per device dirty throttling -v10
Subsystems (22)
cover per device dirty throttling -v10
Per device dirty throttling patches
These patches aim to improve balance_dirty_pages() and directly address three issues:
1) inter device starvation
2) stacked device deadlocks
3) inter process starvation
1 and 2 are a direct result from removing the global dirty limit and using per device dirty limits. By giving each device its own dirty limit is will no longer starve another device, and the cyclic dependancy on the dirty limit is broken.
In order to efficiently distribute the dirty limit across the independant devices a floating proportion is used, this will allocate a share of the total limit proportional to the device's recent activity.
3 is done by also scaling the dirty limit proportional to the current task's recent dirty rate.
Changes since -v9:
- cleaned up the perpcu_counter_init code
- little fixups
- fwd port to .23-rc4-mm1
Changes since -v8:
- cleanup of the proportion code
- fix percpu_counter_add(&counter, -(unsigned long))
- fix per task dirty rate code
- fwd port to .23-rc2-mm2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 1/23 nfs: remove congestion_end()
Its redundant, clear_bdi_congested() already wakes the waiters.
--- fs/nfs/write.c | 5 ++--- include/linux/backing-dev.h | 1 - mm/backing-dev.c | 13 ------------- 3 files changed, 2 insertions(+), 17 deletions(-)
fs/nfs/write.c +1 -3
@@ -235,10 +235,8 @@ static void nfs_end_page_writeback(strucstruct nfs_server *nfss =NFS_SERVER (inode);end_page_writeback (page); -if (atomic_long_dec_return (&nfss->writeback) <NFS_CONGESTION_OFF_THRESH ) { +if (atomic_long_dec_return (&nfss->writeback) <NFS_CONGESTION_OFF_THRESH )clear_bdi_congested (&nfss->backing_dev_info,WRITE ); -congestion_end (WRITE ); - } }/*
include/linux/backing-dev.h +0 -1
@@ -93,7 +93,6 @@ static inline int bdi_rw_congested(strucvoid clear_bdi_congested (struct backing_dev_info *bdi,int rw);void set_bdi_congested (struct backing_dev_info *bdi,int rw);long congestion_wait (int rw,long timeout); -void congestion_end (int rw);#define bdi_cap_writeback_dirty (bdi) \ (!((bdi)->capabilities & BDI_CAP_NO_WRITEBACK))
mm/backing-dev.c +0 -13
@@ -54,16 +54,3 @@ long congestion_wait(int rw, long timeoureturn ret; }EXPORT_SYMBOL (congestion_wait); - -/** -* congestion_end - wake up sleepers on a congested backing_dev_info -* @rw: READ or WRITE -*/ -void congestion_end (int rw) -{ -wait_queue_head_t *wqh = &congestion_wqh[rw]; - -if (waitqueue_active (wqh)) -wake_up (wqh); -} -EXPORT_SYMBOL (congestion_end);
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 2/23 lib: percpu_counter_add
In reply to Peter ZijlstraBecause its a better name, _mod implies modulo.
--- fs/ext2/balloc.c | 4 ++-- fs/ext2/ialloc.c | 2 +- fs/ext3/balloc.c | 4 ++-- fs/ext3/resize.c | 4 ++-- fs/ext4/balloc.c | 4 ++-- fs/ext4/resize.c | 4 ++-- include/linux/percpu_counter.h | 8 ++++---- lib/percpu_counter.c | 4 ++-- 8 files changed, 17 insertions(+), 17 deletions(-)
fs/ext2/balloc.c +2 -2
@@ -138,7 +138,7 @@ static void release_blocks(struct super_if (count) {struct ext2_sb_info *sbi =EXT2_SB (sb); -percpu_counter_mod (&sbi->s_freeblocks_counter, count); +percpu_counter_add (&sbi->s_freeblocks_counter, count); sb->s_dirt =1 ; } }
@@ -1367,7 +1367,7 @@ allocated: }group_adjust_blocks (sb, group_no, gdp, gdp_bh, -num); -percpu_counter_mod (&sbi->s_freeblocks_counter, -num); +percpu_counter_add (&sbi->s_freeblocks_counter, -num);mark_buffer_dirty (bitmap_bh);if (sb->s_flags &MS_SYNCHRONOUS )
fs/ext2/ialloc.c +1 -1
@@ -534,7 +534,7 @@ got: goto fail; } -percpu_counter_mod (&sbi->s_freeinodes_counter,-1 ); +percpu_counter_add (&sbi->s_freeinodes_counter,-1 );if (S_ISDIR (mode))percpu_counter_inc (&sbi->s_dirs_counter);
fs/ext3/balloc.c +2 -2
@@ -609,7 +609,7 @@ do_more:cpu_to_le16 (le16_to_cpu (desc->bg_free_blocks_count) + group_freed);spin_unlock (sb_bgl_lock (sbi, block_group)); -percpu_counter_mod (&sbi->s_freeblocks_counter, count); +percpu_counter_add (&sbi->s_freeblocks_counter, count);/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" );
@@ -1672,7 +1672,7 @@ allocated: gdp->bg_free_blocks_count =cpu_to_le16 (le16_to_cpu (gdp->bg_free_blocks_count)-num);spin_unlock (sb_bgl_lock (sbi, group_no)); -percpu_counter_mod (&sbi->s_freeblocks_counter, -num); +percpu_counter_add (&sbi->s_freeblocks_counter, -num);BUFFER_TRACE (gdp_bh,"journal_dirty_metadata for group descriptor" ); err =ext3_journal_dirty_metadata (handle, gdp_bh);
fs/ext3/resize.c +2 -2
@@ -884,9 +884,9 @@ int ext3_group_add(struct super_block *s input->reserved_blocks);/* Update the free space counts */ -percpu_counter_mod (&sbi->s_freeblocks_counter, +percpu_counter_add (&sbi->s_freeblocks_counter, input->free_blocks_count); -percpu_counter_mod (&sbi->s_freeinodes_counter, +percpu_counter_add (&sbi->s_freeinodes_counter,EXT3_INODES_PER_GROUP (sb));ext3_journal_dirty_metadata (handle, sbi->s_sbh);
fs/ext4/balloc.c +2 -2
@@ -628,7 +628,7 @@ do_more:cpu_to_le16 (le16_to_cpu (desc->bg_free_blocks_count) + group_freed);spin_unlock (sb_bgl_lock (sbi, block_group)); -percpu_counter_mod (&sbi->s_freeblocks_counter, count); +percpu_counter_add (&sbi->s_freeblocks_counter, count);/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" );
@@ -1697,7 +1697,7 @@ allocated: gdp->bg_free_blocks_count =cpu_to_le16 (le16_to_cpu (gdp->bg_free_blocks_count)-num);spin_unlock (sb_bgl_lock (sbi, group_no)); -percpu_counter_mod (&sbi->s_freeblocks_counter, -num); +percpu_counter_add (&sbi->s_freeblocks_counter, -num);BUFFER_TRACE (gdp_bh,"journal_dirty_metadata for group descriptor" ); err =ext4_journal_dirty_metadata (handle, gdp_bh);
fs/ext4/resize.c +2 -2
@@ -893,9 +893,9 @@ int ext4_group_add(struct super_block *s input->reserved_blocks);/* Update the free space counts */ -percpu_counter_mod (&sbi->s_freeblocks_counter, +percpu_counter_add (&sbi->s_freeblocks_counter, input->free_blocks_count); -percpu_counter_mod (&sbi->s_freeinodes_counter, +percpu_counter_add (&sbi->s_freeinodes_counter,EXT4_INODES_PER_GROUP (sb));ext4_journal_dirty_metadata (handle, sbi->s_sbh);
include/linux/percpu_counter.h +4 -4
@@ -32,7 +32,7 @@ struct percpu_counter {void percpu_counter_init (struct percpu_counter *fbc,s64 amount);void percpu_counter_destroy (struct percpu_counter *fbc); -void percpu_counter_mod (struct percpu_counter *fbc,s32 amount); +void percpu_counter_add (struct percpu_counter *fbc,s32 amount);s64 percpu_counter_sum (struct percpu_counter *fbc);static inline s64 percpu_counter_read (struct percpu_counter *fbc)
@@ -71,7 +71,7 @@ static inline void percpu_counter_destro }static inline void -percpu_counter_mod (struct percpu_counter *fbc,s32 amount) +percpu_counter_add (struct percpu_counter *fbc,s32 amount) {preempt_disable (); fbc->count += amount;
@@ -97,12 +97,12 @@ static inline s64 percpu_counter_sum(strstatic inline void percpu_counter_inc (struct percpu_counter *fbc) { -percpu_counter_mod (fbc,1 ); +percpu_counter_add (fbc,1 ); }static inline void percpu_counter_dec (struct percpu_counter *fbc) { -percpu_counter_mod (fbc,-1 ); +percpu_counter_add (fbc,-1 ); }#endif /* _LINUX_PERCPU_COUNTER_H */
lib/percpu_counter.c +2 -2
@@ -14,7 +14,7 @@ static LIST_HEAD(percpu_counters);static DEFINE_MUTEX (percpu_counters_lock);#endif -void percpu_counter_mod (struct percpu_counter *fbc,s32 amount) +void percpu_counter_add (struct percpu_counter *fbc,s32 amount) {long count;s32 *pcount;
@@ -32,7 +32,7 @@ void percpu_counter_mod(struct percpu_co }put_cpu (); } -EXPORT_SYMBOL (percpu_counter_mod); +EXPORT_SYMBOL (percpu_counter_add);/* * Add up all the per-cpu counts, return the result. This is a more accurate
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 3/23 lib: percpu_counter_sub
In reply to Peter ZijlstraHugh spotted that some code does:
percpu_counter_add(&counter, -unsignedlong)
which, when the amount argument is of type s32, sort-of works thanks to two's-complement. However when we'd change the type to s64 this breaks on 32bit machines, because the promotion rules zero extend the unsigned number.
Provide percpu_counter_sub() to hide the s64 cast. That is:
percpu_counter_sub(&counter, foo)
is equal to:
percpu_counter_add(&counter, -(s64)foo);
--- fs/ext2/balloc.c | 2 +- fs/ext3/balloc.c | 2 +- fs/ext4/balloc.c | 2 +- include/linux/percpu_counter.h | 5 +++++ 4 files changed, 8 insertions(+), 3 deletions(-)
fs/ext2/balloc.c +1 -1
@@ -1367,7 +1367,7 @@ allocated: }group_adjust_blocks (sb, group_no, gdp, gdp_bh, -num); -percpu_counter_add (&sbi->s_freeblocks_counter, -num); +percpu_counter_sub (&sbi->s_freeblocks_counter, num);mark_buffer_dirty (bitmap_bh);if (sb->s_flags &MS_SYNCHRONOUS )
fs/ext3/balloc.c +1 -1
@@ -1672,7 +1672,7 @@ allocated: gdp->bg_free_blocks_count =cpu_to_le16 (le16_to_cpu (gdp->bg_free_blocks_count)-num);spin_unlock (sb_bgl_lock (sbi, group_no)); -percpu_counter_add (&sbi->s_freeblocks_counter, -num); +percpu_counter_sub (&sbi->s_freeblocks_counter, num);BUFFER_TRACE (gdp_bh,"journal_dirty_metadata for group descriptor" ); err =ext3_journal_dirty_metadata (handle, gdp_bh);
fs/ext4/balloc.c +1 -1
@@ -1697,7 +1697,7 @@ allocated: gdp->bg_free_blocks_count =cpu_to_le16 (le16_to_cpu (gdp->bg_free_blocks_count)-num);spin_unlock (sb_bgl_lock (sbi, group_no)); -percpu_counter_add (&sbi->s_freeblocks_counter, -num); +percpu_counter_sub (&sbi->s_freeblocks_counter, num);BUFFER_TRACE (gdp_bh,"journal_dirty_metadata for group descriptor" ); err =ext4_journal_dirty_metadata (handle, gdp_bh);
include/linux/percpu_counter.h +5 -0
@@ -105,4 +105,9 @@ static inline void percpu_counter_dec(st percpu_counter_add(fbc, -1); } +static inline void percpu_counter_sub (struct percpu_counter *fbc,s64 amount) +{ +percpu_counter_add (fbc, -amount); +} +#endif /* _LINUX_PERCPU_COUNTER_H */
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 4/23 lib: percpu_counter variable batch
In reply to Peter ZijlstraBecause the current batch setup has an quadric error bound on the counter, allow for an alternative setup.
--- include/linux/percpu_counter.h | 10 +++++++++- lib/percpu_counter.c | 6 +++--- 2 files changed, 12 insertions(+), 4 deletions(-)
include/linux/percpu_counter.h +9 -1
@@ -32,9 +32,14 @@ struct percpu_counter {void percpu_counter_init (struct percpu_counter *fbc,s64 amount);void percpu_counter_destroy (struct percpu_counter *fbc); -void percpu_counter_add (struct percpu_counter *fbc,s32 amount); +void __percpu_counter_add (struct percpu_counter *fbc,s32 amount,s32 batch);s64 percpu_counter_sum (struct percpu_counter *fbc); +static inline void percpu_counter_add (struct percpu_counter *fbc,s32 amount) +{ +__percpu_counter_add (fbc, amount,FBC_BATCH ); +} +static inline s64 percpu_counter_read (struct percpu_counter *fbc) {return fbc->count;
@@ -70,6 +75,9 @@ static inline void percpu_counter_destro { } +#define __percpu_counter_add (fbc, amount, batch) \ + percpu_counter_add(fbc, amount) +static inline void percpu_counter_add (struct percpu_counter *fbc,s32 amount) {
lib/percpu_counter.c +3 -3
@@ -14,7 +14,7 @@ static LIST_HEAD(percpu_counters);static DEFINE_MUTEX (percpu_counters_lock);#endif -void percpu_counter_add (struct percpu_counter *fbc,s32 amount) +void __percpu_counter_add (struct percpu_counter *fbc,s32 amount,s32 batch) {long count;s32 *pcount;
@@ -22,7 +22,7 @@ void percpu_counter_add(struct percpu_co pcount =per_cpu_ptr (fbc->counters, cpu); count = *pcount + amount; -if (count >=FBC_BATCH || count <= -FBC_BATCH ) { +if (count >= batch || count <= -batch) {spin_lock (&fbc->lock); fbc->count += count; *pcount =0 ;
@@ -32,7 +32,7 @@ void percpu_counter_add(struct percpu_co }put_cpu (); } -EXPORT_SYMBOL (percpu_counter_add); +EXPORT_SYMBOL (__percpu_counter_add);/* * Add up all the per-cpu counts, return the result. This is a more accurate
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 5/23 lib: make percpu_counter_add take s64
In reply to Peter Zijlstrapercpu_counter is a s64 counter, make _add consitent.
--- include/linux/percpu_counter.h | 6 +++--- lib/percpu_counter.c | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-)
include/linux/percpu_counter.h +3 -3
@@ -32,10 +32,10 @@ struct percpu_counter {void percpu_counter_init (struct percpu_counter *fbc,s64 amount);void percpu_counter_destroy (struct percpu_counter *fbc); -void __percpu_counter_add (struct percpu_counter *fbc,s32 amount,s32 batch); +void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch);s64 percpu_counter_sum (struct percpu_counter *fbc); -static inline void percpu_counter_add (struct percpu_counter *fbc,s32 amount) +static inline void percpu_counter_add (struct percpu_counter *fbc,s64 amount) {__percpu_counter_add (fbc, amount,FBC_BATCH ); }
@@ -79,7 +79,7 @@ static inline void percpu_counter_destro percpu_counter_add(fbc, amount)static inline void -percpu_counter_add (struct percpu_counter *fbc,s32 amount) +percpu_counter_add (struct percpu_counter *fbc,s64 amount) {preempt_disable (); fbc->count += amount;
lib/percpu_counter.c +2 -2
@@ -14,9 +14,9 @@ static LIST_HEAD(percpu_counters);static DEFINE_MUTEX (percpu_counters_lock);#endif -void __percpu_counter_add (struct percpu_counter *fbc,s32 amount,s32 batch) +void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch) { -long count; +s64 count;s32 *pcount;int cpu =get_cpu ();
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 6/23 lib: percpu_counter_set
In reply to Peter ZijlstraProvide a method to set a percpu counter to a specified value.
--- include/linux/percpu_counter.h | 6 ++++++ lib/percpu_counter.c | 14 ++++++++++++++ 2 files changed, 20 insertions(+)
include/linux/percpu_counter.h +6 -0
@@ -32,6 +32,7 @@ struct percpu_counter {void percpu_counter_init (struct percpu_counter *fbc,s64 amount);void percpu_counter_destroy (struct percpu_counter *fbc); +void percpu_counter_set (struct percpu_counter *fbc,s64 amount);void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch);s64 percpu_counter_sum (struct percpu_counter *fbc);
@@ -75,6 +76,11 @@ static inline void percpu_counter_destro { } +static inline void percpu_counter_set (struct percpu_counter *fbc,s64 amount) +{ + fbc->count = amount; +} +#define __percpu_counter_add (fbc, amount, batch) \ percpu_counter_add(fbc, amount)
lib/percpu_counter.c +14 -0
@@ -14,6 +14,20 @@ static LIST_HEAD(percpu_counters);static DEFINE_MUTEX (percpu_counters_lock);#endif +void percpu_counter_set (struct percpu_counter *fbc,s64 amount) +{ +int cpu; + +spin_lock (&fbc->lock); +for_each_possible_cpu (cpu) { +s32 *pcount =per_cpu_ptr (fbc->counters, cpu); + *pcount =0 ; + } + fbc->count = amount; +spin_unlock (&fbc->lock); +} +EXPORT_SYMBOL (percpu_counter_set); +void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch) {s64 count;
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 7/23 lib: percpu_counter_sum_positive
In reply to Peter ZijlstraBecause its consitent with percpu_counter_read*
--- fs/ext2/super.c | 4 ++-- fs/ext3/super.c | 4 ++-- fs/ext4/balloc.c | 2 +- fs/ext4/super.c | 4 ++-- fs/file_table.c | 2 +- include/linux/percpu_counter.h | 4 ++-- lib/percpu_counter.c | 4 ++-- 7 files changed, 12 insertions(+), 12 deletions(-)
fs/ext3/super.c +2 -2
@@ -2542,13 +2542,13 @@ static int ext3_statfs (struct dentry * buf->f_type =EXT3_SUPER_MAGIC ; buf->f_bsize = sb->s_blocksize; buf->f_blocks =le32_to_cpu (es->s_blocks_count) - sbi->s_overhead_last; - buf->f_bfree =percpu_counter_sum (&sbi->s_freeblocks_counter); + buf->f_bfree =percpu_counter_sum_positive (&sbi->s_freeblocks_counter); es->s_free_blocks_count =cpu_to_le32 (buf->f_bfree); buf->f_bavail = buf->f_bfree -le32_to_cpu (es->s_r_blocks_count);if (buf->f_bfree <le32_to_cpu (es->s_r_blocks_count)) buf->f_bavail =0 ; buf->f_files =le32_to_cpu (es->s_inodes_count); - buf->f_ffree =percpu_counter_sum (&sbi->s_freeinodes_counter); + buf->f_ffree =percpu_counter_sum_positive (&sbi->s_freeinodes_counter); es->s_free_inodes_count =cpu_to_le32 (buf->f_ffree); buf->f_namelen =EXT3_NAME_LEN ; fsid = le64_to_cpup((void *)es->s_uuid) ^
fs/ext4/super.c +2 -2
@@ -2692,13 +2692,13 @@ static int ext4_statfs (struct dentry * buf->f_type =EXT4_SUPER_MAGIC ; buf->f_bsize = sb->s_blocksize; buf->f_blocks =ext4_blocks_count (es) - sbi->s_overhead_last; - buf->f_bfree =percpu_counter_sum (&sbi->s_freeblocks_counter); + buf->f_bfree =percpu_counter_sum_positive (&sbi->s_freeblocks_counter); es->s_free_blocks_count =cpu_to_le32 (buf->f_bfree); buf->f_bavail = buf->f_bfree -ext4_r_blocks_count (es);if (buf->f_bfree <ext4_r_blocks_count (es)) buf->f_bavail =0 ; buf->f_files =le32_to_cpu (es->s_inodes_count); - buf->f_ffree =percpu_counter_sum (&sbi->s_freeinodes_counter); + buf->f_ffree =percpu_counter_sum_positive (&sbi->s_freeinodes_counter); es->s_free_inodes_count =cpu_to_le32 (buf->f_ffree); buf->f_namelen =EXT4_NAME_LEN ; fsid = le64_to_cpup((void *)es->s_uuid) ^
fs/file_table.c +1 -1
@@ -98,7 +98,7 @@ struct file *get_empty_filp(void)* percpu_counters are inaccurate. Do an expensive check before * we go and fail. */ -if (percpu_counter_sum (&nr_files) >= files_stat.max_files) +if (percpu_counter_sum_positive (&nr_files) >= files_stat.max_files) goto over; }
include/linux/percpu_counter.h +2 -2
@@ -34,7 +34,7 @@ void percpu_counter_init(struct percpu_cvoid percpu_counter_destroy (struct percpu_counter *fbc);void percpu_counter_set (struct percpu_counter *fbc,s64 amount);void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch); -s64 percpu_counter_sum (struct percpu_counter *fbc); +s64 percpu_counter_sum_positive (struct percpu_counter *fbc);static inline void percpu_counter_add (struct percpu_counter *fbc,s64 amount) {
@@ -102,7 +102,7 @@ static inline s64 percpu_counter_read_poreturn fbc->count; } -static inline s64 percpu_counter_sum (struct percpu_counter *fbc) +static inline s64 percpu_counter_sum_positive (struct percpu_counter *fbc) {return percpu_counter_read_positive (fbc); }
lib/percpu_counter.c +2 -2
@@ -52,7 +52,7 @@ EXPORT_SYMBOL(__percpu_counter_add);* Add up all the per-cpu counts, return the result. This is a more accurate * but much slower version of percpu_counter_read_positive() */ -s64 percpu_counter_sum (struct percpu_counter *fbc) +s64 percpu_counter_sum_positive (struct percpu_counter *fbc) {s64 ret;int cpu;
@@ -66,7 +66,7 @@ s64 percpu_counter_sum(struct percpu_couspin_unlock (&fbc->lock);return ret <0 ?0 : ret; } -EXPORT_SYMBOL (percpu_counter_sum); +EXPORT_SYMBOL (percpu_counter_sum_positive);void percpu_counter_init (struct percpu_counter *fbc,s64 amount) {
fs/ext2/super.c +2 -2
@@ -1232,13 +1232,13 @@ static int ext2_statfs (struct dentry * buf->f_type =EXT2_SUPER_MAGIC ; buf->f_bsize = sb->s_blocksize; buf->f_blocks =le32_to_cpu (es->s_blocks_count) - sbi->s_overhead_last; - buf->f_bfree =percpu_counter_sum (&sbi->s_freeblocks_counter); + buf->f_bfree =percpu_counter_sum_positive (&sbi->s_freeblocks_counter); es->s_free_blocks_count =cpu_to_le32 (buf->f_bfree); buf->f_bavail = buf->f_bfree -le32_to_cpu (es->s_r_blocks_count);if (buf->f_bfree <le32_to_cpu (es->s_r_blocks_count)) buf->f_bavail =0 ; buf->f_files =le32_to_cpu (es->s_inodes_count); - buf->f_ffree =percpu_counter_sum (&sbi->s_freeinodes_counter); + buf->f_ffree =percpu_counter_sum_positive (&sbi->s_freeinodes_counter); es->s_free_inodes_count =cpu_to_le32 (buf->f_ffree); buf->f_namelen =EXT2_NAME_LEN ; fsid = le64_to_cpup((void *)es->s_uuid) ^
fs/ext4/balloc.c +1 -1
@@ -2020,7 +2020,7 @@ int ext4_reserve_init(struct super_blockspin_lock_init (&rs[i].rs_lock); rs[i].rs_reserved =0 ; } - rs[0 ].rs_reserved =percpu_counter_sum (&sbi->s_freeblocks_counter); + rs[0 ].rs_reserved =percpu_counter_sum_positive (&sbi->s_freeblocks_counter);return 0 ; }
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 8/23 lib: percpu_count_sum()
In reply to Peter ZijlstraProvide an accurate version of percpu_counter_read.
--- include/linux/percpu_counter.h | 18 +++++++++++++++++- lib/percpu_counter.c | 6 +++--- 2 files changed, 20 insertions(+), 4 deletions(-)
include/linux/percpu_counter.h +17 -1
@@ -34,13 +34,24 @@ void percpu_counter_init(struct percpu_cvoid percpu_counter_destroy (struct percpu_counter *fbc);void percpu_counter_set (struct percpu_counter *fbc,s64 amount);void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch); -s64 percpu_counter_sum_positive (struct percpu_counter *fbc); +s64 __percpu_counter_sum (struct percpu_counter *fbc);static inline void percpu_counter_add (struct percpu_counter *fbc,s64 amount) {__percpu_counter_add (fbc, amount,FBC_BATCH ); } +static inline s64 percpu_counter_sum_positive (struct percpu_counter *fbc) +{ +s64 ret =__percpu_counter_sum (fbc); +return ret <0 ?0 : ret; +} + +static inline s64 percpu_counter_sum (struct percpu_counter *fbc) +{ +return __percpu_counter_sum (fbc); +} +static inline s64 percpu_counter_read (struct percpu_counter *fbc) {return fbc->count;
@@ -107,6 +118,11 @@ static inline s64 percpu_counter_sum_pos return percpu_counter_read_positive(fbc); } +static inline s64 percpu_counter_sum (struct percpu_counter *fbc) +{ +return percpu_counter_read (fbc); +} +#endif /* CONFIG_SMP */ static inline void percpu_counter_inc (struct percpu_counter *fbc)
lib/percpu_counter.c +3 -3
@@ -52,7 +52,7 @@ EXPORT_SYMBOL(__percpu_counter_add);* Add up all the per-cpu counts, return the result. This is a more accurate * but much slower version of percpu_counter_read_positive() */ -s64 percpu_counter_sum_positive (struct percpu_counter *fbc) +s64 __percpu_counter_sum (struct percpu_counter *fbc) {s64 ret;int cpu;
@@ -64,9 +64,9 @@ s64 percpu_counter_sum_positive(struct p ret += *pcount; }spin_unlock (&fbc->lock); -return ret <0 ?0 : ret; +return ret; } -EXPORT_SYMBOL (percpu_counter_sum_positive); +EXPORT_SYMBOL (__percpu_counter_sum);void percpu_counter_init (struct percpu_counter *fbc,s64 amount) {
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 9/23 lib: percpu_counter_init error handling
In reply to Peter Zijlstraalloc_percpu can fail, propagate that error.
--- fs/ext2/super.c | 15 ++++++++++++--- fs/ext3/super.c | 21 +++++++++++++++------ fs/ext4/super.c | 21 +++++++++++++++------ include/linux/percpu_counter.h | 5 +++-- lib/percpu_counter.c | 8 +++++++- 5 files changed, 52 insertions(+), 18 deletions(-)
fs/ext2/super.c +12 -3
@@ -725,6 +725,7 @@ static int ext2_fill_super(struct super_int db_count;int i, j;__le32 features; +int err; sbi =kzalloc (sizeof (*sbi),GFP_KERNEL );if (!sbi)
@@ -996,12 +997,20 @@ static int ext2_fill_super(struct super_ sbi->s_rsv_window_head.rsv_goal_size =0 ;ext2_rsv_window_add (sb, &sbi->s_rsv_window_head); -percpu_counter_init (&sbi->s_freeblocks_counter, + err =percpu_counter_init (&sbi->s_freeblocks_counter,ext2_count_free_blocks (sb)); -percpu_counter_init (&sbi->s_freeinodes_counter, +if (!err) { + err =percpu_counter_init (&sbi->s_freeinodes_counter,ext2_count_free_inodes (sb)); -percpu_counter_init (&sbi->s_dirs_counter, + } +if (!err) { + err =percpu_counter_init (&sbi->s_dirs_counter,ext2_count_dirs (sb)); + } +if (err) { +printk (KERN_ERR "EXT2-fs: insufficient memory\n" ); + goto failed_mount3; + }/* * set up enough so that it can read an inode */
fs/ext3/super.c +15 -6
@@ -1485,6 +1485,7 @@ static int ext3_fill_super (struct superint i;int needs_recovery;__le32 features; +int err; sbi =kzalloc (sizeof (*sbi),GFP_KERNEL );if (!sbi)
@@ -1745,12 +1746,20 @@ static int ext3_fill_super (struct superget_random_bytes (&sbi->s_next_generation,sizeof (u32));spin_lock_init (&sbi->s_next_gen_lock); -percpu_counter_init (&sbi->s_freeblocks_counter, -ext3_count_free_blocks (sb)); -percpu_counter_init (&sbi->s_freeinodes_counter, -ext3_count_free_inodes (sb)); -percpu_counter_init (&sbi->s_dirs_counter, -ext3_count_dirs (sb)); + err =percpu_counter_init (&sbi->s_freeblocks_counter, +ext3_count_free_blocks (sb)); +if (!err) { + err =percpu_counter_init (&sbi->s_freeinodes_counter, +ext3_count_free_inodes (sb)); + } +if (!err) { + err =percpu_counter_init (&sbi->s_dirs_counter, +ext3_count_dirs (sb)); + } +if (err) { +printk (KERN_ERR "EXT3-fs: insufficient memory\n" ); + goto failed_mount3; + }/* per fileystem reservation list head & lock */ spin_lock_init (&sbi->s_rsv_window_lock);
fs/ext4/super.c +15 -6
@@ -1576,6 +1576,7 @@ static int ext4_fill_super (struct superint needs_recovery;__le32 features;__u64 blocks_count; +int err; sbi =kzalloc (sizeof (*sbi),GFP_KERNEL );if (!sbi)
@@ -1857,12 +1858,20 @@ static int ext4_fill_super (struct superget_random_bytes (&sbi->s_next_generation,sizeof (u32));spin_lock_init (&sbi->s_next_gen_lock); -percpu_counter_init (&sbi->s_freeblocks_counter, -ext4_count_free_blocks (sb)); -percpu_counter_init (&sbi->s_freeinodes_counter, -ext4_count_free_inodes (sb)); -percpu_counter_init (&sbi->s_dirs_counter, -ext4_count_dirs (sb)); + err =percpu_counter_init (&sbi->s_freeblocks_counter, +ext4_count_free_blocks (sb)); +if (!err) { + err =percpu_counter_init (&sbi->s_freeinodes_counter, +ext4_count_free_inodes (sb)); + } +if (!err) { + err =percpu_counter_init (&sbi->s_dirs_counter, +ext4_count_dirs (sb)); + } +if (err) { +printk (KERN_ERR "EXT4-fs: insufficient memory\n" ); + goto failed_mount3; + }/* per fileystem reservation list head & lock */ spin_lock_init (&sbi->s_rsv_window_lock);
include/linux/percpu_counter.h +3 -2
@@ -30,7 +30,7 @@ struct percpu_counter {#define FBC_BATCH (NR_CPUS*4)#endif -void percpu_counter_init (struct percpu_counter *fbc,s64 amount); +int percpu_counter_init (struct percpu_counter *fbc,s64 amount);void percpu_counter_destroy (struct percpu_counter *fbc);void percpu_counter_set (struct percpu_counter *fbc,s64 amount);void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch);
@@ -78,9 +78,10 @@ struct percpu_counter {s64 count; }; -static inline void percpu_counter_init (struct percpu_counter *fbc,s64 amount) +static inline int percpu_counter_init (struct percpu_counter *fbc,s64 amount) { fbc->count = amount; +return 0 ; }static inline void percpu_counter_destroy (struct percpu_counter *fbc)
lib/percpu_counter.c +7 -1
@@ -68,21 +68,27 @@ s64 __percpu_counter_sum(struct percpu_c }EXPORT_SYMBOL (__percpu_counter_sum); -void percpu_counter_init (struct percpu_counter *fbc,s64 amount) +int percpu_counter_init (struct percpu_counter *fbc,s64 amount) {spin_lock_init (&fbc->lock); fbc->count = amount; fbc->counters =alloc_percpu (s32); +if (!fbc->counters) +return -ENOMEM ;#ifdef CONFIG_HOTPLUG_CPU mutex_lock (&percpu_counters_lock);list_add (&fbc->list, &percpu_counters);mutex_unlock (&percpu_counters_lock);#endif +return 0 ; }EXPORT_SYMBOL (percpu_counter_init);void percpu_counter_destroy (struct percpu_counter *fbc) { +if (!fbc->counters) +return ; +free_percpu (fbc->counters);#ifdef CONFIG_HOTPLUG_CPU mutex_lock (&percpu_counters_lock);
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 10/23 lib: percpu_counter_init_irq
In reply to Peter Zijlstraprovide a way to tell lockdep about percpu_counters that are supposed to be used from irq safe contexts.
--- include/linux/percpu_counter.h | 3 +++ lib/percpu_counter.c | 12 ++++++++++++ 2 files changed, 15 insertions(+)
include/linux/percpu_counter.h +3 -0
@@ -31,6 +31,7 @@ struct percpu_counter {#endif int percpu_counter_init (struct percpu_counter *fbc,s64 amount); +int percpu_counter_init_irq (struct percpu_counter *fbc,s64 amount);void percpu_counter_destroy (struct percpu_counter *fbc);void percpu_counter_set (struct percpu_counter *fbc,s64 amount);void __percpu_counter_add (struct percpu_counter *fbc,s64 amount,s32 batch);
@@ -84,6 +85,8 @@ static inline int percpu_counter_init(streturn 0 ; } +#define percpu_counter_init_irq percpu_counter_init +static inline void percpu_counter_destroy (struct percpu_counter *fbc) { }
lib/percpu_counter.c +12 -0
@@ -68,6 +68,8 @@ s64 __percpu_counter_sum(struct percpu_c }EXPORT_SYMBOL (__percpu_counter_sum); +static struct lock_class_key percpu_counter_irqsafe; +int percpu_counter_init (struct percpu_counter *fbc,s64 amount) {spin_lock_init (&fbc->lock);
@@ -84,6 +86,16 @@ int percpu_counter_init(struct percpu_co }EXPORT_SYMBOL (percpu_counter_init); +int percpu_counter_init_irq (struct percpu_counter *fbc,s64 amount) +{ +int err; + + err =percpu_counter_init (fbc, amount); +if (!err) +lockdep_set_class (&fbc->lock, &percpu_counter_irqsafe); +return err; +} +void percpu_counter_destroy (struct percpu_counter *fbc) {if (!fbc->counters)
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 11/23 mm: bdi init hooks
In reply to Peter Zijlstraprovide BDI constructor/destructor hooks
--- block/ll_rw_blk.c | 13 ++++++++++--- drivers/block/rd.c | 20 +++++++++++++++++++- drivers/char/mem.c | 5 +++++ fs/char_dev.c | 1 + fs/configfs/configfs_internal.h | 2 ++ fs/configfs/inode.c | 8 ++++++++ fs/configfs/mount.c | 9 +++++++++ fs/fuse/inode.c | 9 +++++++++ fs/hugetlbfs/inode.c | 9 ++++++++- fs/nfs/client.c | 6 ++++++ fs/ocfs2/dlm/dlmfs.c | 9 ++++++++- fs/ramfs/inode.c | 12 +++++++++++- fs/sysfs/inode.c | 5 +++++ fs/sysfs/mount.c | 4 ++++ fs/sysfs/sysfs.h | 1 + include/linux/backing-dev.h | 8 ++++++++ mm/readahead.c | 6 ++++++ mm/shmem.c | 6 ++++++ mm/swap.c | 5 +++++ 19 files changed, 131 insertions(+), 7 deletions(-)
block/ll_rw_blk.c +10 -3
@@ -1784,6 +1784,7 @@ static void blk_release_queue(struct kobblk_trace_shutdown (q); +bdi_destroy (&q->backing_dev_info);kmem_cache_free (requestq_cachep, q); }
@@ -1837,21 +1838,27 @@ static struct kobj_type queue_ktype;struct request_queue *blk_alloc_queue_node (gfp_t gfp_mask,int node_id) {struct request_queue *q; +int err; q =kmem_cache_alloc_node (requestq_cachep, gfp_mask | __GFP_ZERO, node_id);if (!q)return NULL ; + q->backing_dev_info.unplug_io_fn = blk_backing_dev_unplug; + q->backing_dev_info.unplug_io_data = q; + err =bdi_init (&q->backing_dev_info); +if (err) { +kmem_cache_free (requestq_cachep, q); +return NULL ; + } +init_timer (&q->unplug_timer);snprintf (q->kobj.name,KOBJ_NAME_LEN ,"%s" ,"queue" ); q->kobj.ktype = &queue_ktype;kobject_init (&q->kobj); - q->backing_dev_info.unplug_io_fn = blk_backing_dev_unplug; - q->backing_dev_info.unplug_io_data = q; -mutex_init (&q->sysfs_lock);return q;
drivers/block/rd.c +19 -1
@@ -411,6 +411,9 @@ static void __exit rd_cleanup(void)blk_cleanup_queue (rd_queue[i]); }unregister_blkdev (RAMDISK_MAJOR ,"ramdisk" ); + +bdi_destroy (&rd_file_backing_dev_info); +bdi_destroy (&rd_backing_dev_info); }/*
@@ -419,7 +422,19 @@ static void __exit rd_cleanup(void)static int __init rd_init (void ) {int i; -int err = -ENOMEM ; +int err; + + err =bdi_init (&rd_backing_dev_info); +if (err) + goto out2; + + err =bdi_init (&rd_file_backing_dev_info); +if (err) { +bdi_destroy (&rd_backing_dev_info); + goto out2; + } + + err = -ENOMEM ;if (rd_blocksize >PAGE_SIZE || rd_blocksize <512 || (rd_blocksize & (rd_blocksize-1 ))) {
@@ -473,6 +488,9 @@ out:put_disk (rd_disks[i]);blk_cleanup_queue (rd_queue[i]); } +bdi_destroy (&rd_backing_dev_info); +bdi_destroy (&rd_file_backing_dev_info); +out2:return err; }
drivers/char/mem.c +5 -0
@@ -893,6 +893,11 @@ static struct class *mem_class;static int __init chr_dev_init (void ) {int i; +int err; + + err =bdi_init (&zero_bdi); +if (err) +return err;if (register_chrdev (MEM_MAJOR ,"mem" ,&memory_fops))printk ("unable to get major %d for memory devs\n" ,MEM_MAJOR );
fs/char_dev.c +1 -0
@@ -545,6 +545,7 @@ static struct kobject *base_probe(dev_tvoid __init chrdev_init (void ) { cdev_map =kobj_map_init (base_probe, &chrdevs_lock); +bdi_init (&directly_mappable_cdev_bdi); }
fs/fuse/inode.c +9 -0
@@ -444,6 +444,7 @@ static int fuse_show_options(struct seq_static struct fuse_conn *new_conn (void ) {struct fuse_conn *fc; +int err; fc =kzalloc (sizeof (*fc),GFP_KERNEL );if (fc) {
@@ -460,10 +461,17 @@ static struct fuse_conn *new_conn(void)atomic_set (&fc->num_waiting,0 ); fc->bdi.ra_pages = (VM_MAX_READAHEAD *1024 ) /PAGE_CACHE_SIZE ; fc->bdi.unplug_io_fn = default_unplug_io_fn; + err =bdi_init (&fc->bdi); +if (err) { +kfree (fc); + fc =NULL ; + goto out; + } fc->reqctr =0 ; fc->blocked =1 ;get_random_bytes (&fc->scramble_key,sizeof (fc->scramble_key)); } +out:return fc; }
@@ -473,6 +481,7 @@ void fuse_conn_put(struct fuse_conn *fc)if (fc->destroy_req)fuse_request_free (fc->destroy_req);mutex_destroy (&fc->inst_mutex); +bdi_destroy (&fc->bdi);kfree (fc); } }
fs/nfs/client.c +6 -0
@@ -632,6 +632,7 @@ static void nfs_server_set_fsinfo(structif (server->rsize >NFS_MAX_FILE_IO_SIZE ) server->rsize =NFS_MAX_FILE_IO_SIZE ; server->rpages = (server->rsize +PAGE_CACHE_SIZE -1 ) >>PAGE_CACHE_SHIFT ; + server->backing_dev_info.ra_pages = server->rpages *NFS_MAX_READAHEAD ;if (server->wsize > max_rpc_payload)
@@ -682,6 +683,10 @@ static int nfs_probe_fsinfo(struct nfs_s goto out_error;nfs_server_set_fsinfo (server, &fsinfo); + error =bdi_init (&server->backing_dev_info); +if (error) + goto out_error; +/* Get some general file system info */ if (server->namelen ==0 ) {
@@ -761,6 +766,7 @@ void nfs_free_server(struct nfs_server *nfs_put_client (server->nfs_client);nfs_free_iostats (server->io_stats); +bdi_destroy (&server->backing_dev_info);kfree (server);nfs_release_automount_timer ();dprintk ("<-- nfs_free_server()\n" );
include/linux/backing-dev.h +8 -0
@@ -34,6 +34,14 @@ struct backing_dev_info {void *unplug_io_data; }; +static inline int bdi_init (struct backing_dev_info *bdi) +{ +return 0 ; +} + +static inline void bdi_destroy (struct backing_dev_info *bdi) +{ +}/* * Flags in backing_dev_info::capability
fs/hugetlbfs/inode.c +8 -1
@@ -964,11 +964,15 @@ static int __init init_hugetlbfs_fs(voidint error;struct vfsmount *vfsmount; + error =bdi_init (&hugetlbfs_backing_dev_info); +if (error) +return error; + hugetlbfs_inode_cachep =kmem_cache_create ("hugetlbfs_inode_cache" ,sizeof (struct hugetlbfs_inode_info ),0 ,0 , init_once);if (hugetlbfs_inode_cachep ==NULL ) -return -ENOMEM ; + goto out2; error =register_filesystem (&hugetlbfs_fs_type);if (error)
@@ -986,6 +990,8 @@ static int __init init_hugetlbfs_fs(void out:if (error)kmem_cache_destroy (hugetlbfs_inode_cachep); + out2: +bdi_destroy (&hugetlbfs_backing_dev_info);return error; }
@@ -993,6 +999,7 @@ static void __exit exit_hugetlbfs_fs(voi {kmem_cache_destroy (hugetlbfs_inode_cachep);unregister_filesystem (&hugetlbfs_fs_type); +bdi_destroy (&hugetlbfs_backing_dev_info); }module_init (init_hugetlbfs_fs)
fs/ocfs2/dlm/dlmfs.c +8 -1
@@ -588,13 +588,17 @@ static int __init init_dlmfs_fs(void)dlmfs_print_version (); + status =bdi_init (&dlmfs_backing_dev_info); +if (status) +return status; + dlmfs_inode_cache =kmem_cache_create ("dlmfs_inode_cache" ,sizeof (struct dlmfs_inode_private ),0 , (SLAB_HWCACHE_ALIGN |SLAB_RECLAIM_ACCOUNT |SLAB_MEM_SPREAD ), dlmfs_init_once);if (!dlmfs_inode_cache) -return -ENOMEM ; + goto bail; cleanup_inode =1 ; user_dlm_worker =create_singlethread_workqueue ("user_dlm" );
@@ -611,6 +615,7 @@ bail:kmem_cache_destroy (dlmfs_inode_cache);if (cleanup_worker)destroy_workqueue (user_dlm_worker); +bdi_destroy (&dlmfs_backing_dev_info); }else printk ("OCFS2 User DLM kernel interface loaded\n" );return status;
@@ -624,6 +629,8 @@ static void __exit exit_dlmfs_fs(void)destroy_workqueue (user_dlm_worker);kmem_cache_destroy (dlmfs_inode_cache); + +bdi_destroy (&dlmfs_backing_dev_info); }MODULE_AUTHOR ("Oracle" );
fs/configfs/configfs_internal.h +2 -0
@@ -56,6 +56,8 @@ extern int configfs_is_root(struct confiextern struct inode *configfs_new_inode (mode_t mode,struct configfs_dirent *);extern int configfs_create (struct dentry *,int mode,int (*init)(struct inode *)); +extern int configfs_inode_init (void ); +extern void configfs_inode_exit (void );extern int configfs_create_file (struct config_item *,const struct configfs_attribute *); extern int configfs_make_dirent(struct configfs_dirent *,
fs/configfs/inode.c +8 -0
@@ -256,4 +256,12 @@ void configfs_hash_and_remove(struct denmutex_unlock (&dir->d_inode->i_mutex); } +int __init configfs_inode_init (void ) +{ +return bdi_init (&configfs_backing_dev_info); +} +void __exit configfs_inode_exit(void) +{ +bdi_destroy (&configfs_backing_dev_info); +}
fs/configfs/mount.c +9 -0
@@ -154,8 +154,16 @@ static int __init configfs_init(void)subsystem_unregister (&config_subsys);kmem_cache_destroy (configfs_dir_cachep); configfs_dir_cachep =NULL ; + goto out; } + err =configfs_inode_init (); +if (err) { +unregister_filesystem (&configfs_fs_type); +subsystem_unregister (&config_subsys); +kmem_cache_destroy (configfs_dir_cachep); + configfs_dir_cachep =NULL ; + } out:return err; }
@@ -166,6 +174,7 @@ static void __exit configfs_exit(void)subsystem_unregister (&config_subsys);kmem_cache_destroy (configfs_dir_cachep); configfs_dir_cachep =NULL ; +configfs_inode_exit (); }MODULE_AUTHOR ("Oracle" );
fs/ramfs/inode.c +11 -1
@@ -223,7 +223,17 @@ module_exit(exit_ramfs_fs)int __init init_rootfs (void ) { -return register_filesystem (&rootfs_fs_type); +int err; + + err =bdi_init (&ramfs_backing_dev_info); +if (err) +return err; + + err =register_filesystem (&rootfs_fs_type); +if (err) +bdi_destroy (&ramfs_backing_dev_info); + +return err; }MODULE_LICENSE ("GPL" );
fs/sysfs/inode.c +5 -0
@@ -33,6 +33,11 @@ static const struct inode_operations sys .setattr = sysfs_setattr, }; +int __init sysfs_inode_init (void ) +{ +return bdi_init (&sysfs_backing_dev_info); +} +int sysfs_setattr (struct dentry * dentry,struct iattr * iattr) {struct inode * inode = dentry->d_inode;
fs/sysfs/mount.c +4 -0
@@ -83,6 +83,10 @@ int __init sysfs_init(void)if (!sysfs_dir_cachep) goto out; + err =sysfs_inode_init (); +if (err) + goto out_err; + err =register_filesystem (&sysfs_fs_type);if (!err) { sysfs_mount =kern_mount (&sysfs_fs_type);
fs/sysfs/sysfs.h +1 -0
@@ -68,6 +68,7 @@ extern void sysfs_remove_one(struct sysfextern void sysfs_addrm_finish (struct sysfs_addrm_cxt *acxt);extern struct inode *sysfs_get_inode (struct sysfs_dirent *sd); +extern int sysfs_inode_init (void );extern void release_sysfs_dirent (struct sysfs_dirent * sd); extern struct sysfs_dirent *sysfs_find_dirent(struct sysfs_dirent *parent_sd,
mm/shmem.c +6 -0
@@ -2462,6 +2462,10 @@ static int __init init_tmpfs(void) {int error; + error =bdi_init (&shmem_backing_dev_info); +if (error) + goto out4; + error =init_inodecache ();if (error) goto out3;
@@ -2486,6 +2490,8 @@ out1: out2:destroy_inodecache (); out3: +bdi_destroy (&shmem_backing_dev_info); +out4: shm_mnt =ERR_PTR (error);return error; }
mm/swap.c +5 -0
@@ -29,6 +29,7 @@#include <linux/cpu.h> #include <linux/notifier.h> #include <linux/memcontrol.h> +#include <linux/backing-dev.h> /* How many pages do we try to swap or page in/out together? */ int page_cluster;
@@ -505,6 +506,10 @@ void __init swap_setup(void) {unsigned long megs = num_physpages >> (20 -PAGE_SHIFT ); +#ifdef CONFIG_SWAP +bdi_init (swapper_space.backing_dev_info); +#endif +/* Use a smaller cluster for small-memory machines */ if (megs <16 ) page_cluster =2 ;
mm/readahead.c +6 -0
@@ -232,6 +232,12 @@ unsigned long max_sane_readahead(unsigne + node_page_state(numa_node_id(), NR_FREE_PAGES)) / 2); } +static int __init readahead_init (void ) +{ +return bdi_init (&default_backing_dev_info); +} +subsys_initcall (readahead_init); +/* * Submit IO for the read-ahead request in file_ra_state. */
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 12/23 containers: bdi init hooks
In reply to Peter Zijlstrasplit off from the large bdi_init patch because containers are not slated for mainline any time soon.
--- kernel/container.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
kernel/container.c +11 -3
@@ -567,12 +567,13 @@ static int container_populate_dir(structstatic struct inode_operations container_dir_inode_operations;static struct file_operations proc_containerstats_operations; +static struct backing_dev_info container_backing_dev_info = { + .capabilities =BDI_CAP_NO_ACCT_DIRTY |BDI_CAP_NO_WRITEBACK , +}; +static struct inode *container_new_inode (mode_t mode,struct super_block *sb) {struct inode *inode =new_inode (sb); -static struct backing_dev_info container_backing_dev_info = { - .capabilities =BDI_CAP_NO_ACCT_DIRTY |BDI_CAP_NO_WRITEBACK , - };if (inode) { inode->i_mode = mode;
@@ -2261,6 +2262,10 @@ int __init container_init(void)int i;struct proc_dir_entry *entry; + err =bdi_init (&container_backing_dev_info); +if (err) +return err; +for (i =0 ; i <CONTAINER_SUBSYS_COUNT ; i++) {struct container_subsys *ss = subsys[i];if (!ss->early_init)
@@ -2276,6 +2281,9 @@ int __init container_init(void) entry->proc_fops = &proc_containerstats_operations; out: +if (err) +bdi_destroy (&container_backing_dev_info); +return err; }
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 13/23 mtd: bdi init hooks
In reply to Peter Zijlstrasplit off because the relevant mtd changes seem particular to -mm
--- drivers/mtd/mtdcore.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/mtd/mtdcore.c +9 -0
@@ -48,6 +48,7 @@ static LIST_HEAD(mtd_notifiers);int add_mtd_device (struct mtd_info *mtd) {int i; +int err;if (!mtd->backing_dev_info) {switch (mtd->type) {
@@ -62,6 +63,9 @@ int add_mtd_device(struct mtd_info *mtd)break ; } } + err =bdi_init (mtd->backing_dev_info); +if (err) +return 1 ;BUG_ON (mtd->writesize ==0 );mutex_lock (&mtd_table_mutex);
@@ -102,6 +106,7 @@ int add_mtd_device(struct mtd_info *mtd) }mutex_unlock (&mtd_table_mutex); +bdi_destroy (mtd->backing_dev_info);return 1 ; }
@@ -144,6 +149,10 @@ int del_mtd_device (struct mtd_info *mtd }mutex_unlock (&mtd_table_mutex); + +if (mtd->backing_dev_info) +bdi_destroy (mtd->backing_dev_info); +return ret; }
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 14/23 mtd: clean up the backing_dev_info usage
In reply to Peter ZijlstraGive each mtd device its own backing_dev_info instance.
--- drivers/mtd/mtdcore.c | 8 +++++--- include/linux/mtd/mtd.h | 2 ++ 2 files changed, 7 insertions(+), 3 deletions(-)
drivers/mtd/mtdcore.c +5 -3
@@ -19,6 +19,7 @@#include <linux/init.h> #include <linux/mtd/compatmac.h> #include <linux/proc_fs.h> +#include <linux/backing-dev.h> #include <linux/mtd/mtd.h> #include "internal.h"
@@ -53,15 +54,16 @@ int add_mtd_device(struct mtd_info *mtd)if (!mtd->backing_dev_info) {switch (mtd->type) {case MTD_RAM : - mtd->backing_dev_info = &mtd_bdi_rw_mappable; + mtd->mtd_backing_dev_info = mtd_bdi_rw_mappable;break ;case MTD_ROM : - mtd->backing_dev_info = &mtd_bdi_ro_mappable; + mtd->mtd_backing_dev_info = mtd_bdi_ro_mappable;break ;default : - mtd->backing_dev_info = &mtd_bdi_unmappable; + mtd->mtd_backing_dev_info = mtd_bdi_unmappable;break ; } + mtd->backing_dev_info = &mtd->mtd_backing_dev_info; } err =bdi_init (mtd->backing_dev_info);if (err)
include/linux/mtd/mtd.h +2 -0
@@ -13,6 +13,7 @@#include <linux/module.h> #include <linux/uio.h> #include <linux/notifier.h> +#include <linux/backing-dev.h> #include <linux/mtd/compatmac.h> #include <mtd/mtd-abi.h>
@@ -154,6 +155,7 @@ struct mtd_info {* - provides mmap capabilities */ struct backing_dev_info *backing_dev_info; +struct backing_dev_info mtd_backing_dev_info;int (*read) (struct mtd_info *mtd,loff_t from,size_t len,size_t *retlen,u_char *buf);
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 15/23 mtd: give mtdconcat devices their own backing_dev_info
In reply to Peter ZijlstraThese are actual devices, give them their own BDI.
--- drivers/mtd/mtdconcat.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-)
drivers/mtd/mtdconcat.c +18 -10
@@ -32,6 +32,7 @@ struct mtd_concat {struct mtd_info mtd;int num_subdev;struct mtd_info **subdev; +struct backing_dev_info backing_dev_info; };/*
@@ -782,10 +783,9 @@ struct mtd_info *mtd_concat_create(strucfor (i =1 ; i < num_devs; i++) {if (concat->mtd.type != subdev[i]->type) { -kfree (concat);printk ("Incompatible device type on \"%s\"\n" , subdev[i]->name); -return NULL ; + goto error; }if (concat->mtd.flags != subdev[i]->flags) {/*
@@ -794,10 +794,9 @@ struct mtd_info *mtd_concat_create(struc*/ if ((concat->mtd.flags ^ subdev[i]-> flags) & ~MTD_WRITEABLE ) { -kfree (concat);printk ("Incompatible device flags on \"%s\"\n" , subdev[i]->name); -return NULL ; + goto error; }else /* if writeable attribute differs, make super device writeable */
@@ -809,9 +808,12 @@ struct mtd_info *mtd_concat_create(struc* - copy-mapping is still permitted */ if (concat->mtd.backing_dev_info != - subdev[i]->backing_dev_info) + subdev[i]->backing_dev_info) { + concat->backing_dev_info = default_backing_dev_info; +bdi_init (&concat->backing_dev_info); concat->mtd.backing_dev_info = - &default_backing_dev_info; + &concat->backing_dev_info; + } concat->mtd.size += subdev[i]->size; concat->mtd.ecc_stats.badblocks +=
@@ -821,10 +823,9 @@ struct mtd_info *mtd_concat_create(struc concat->mtd.oobsize != subdev[i]->oobsize || !concat->mtd.read_oob != !subdev[i]->read_oob || !concat->mtd.write_oob != !subdev[i]->write_oob) { -kfree (concat);printk ("Incompatible OOB or ECC data on \"%s\"\n" , subdev[i]->name); -return NULL ; + goto error; } concat->subdev[i] = subdev[i];
@@ -903,11 +904,10 @@ struct mtd_info *mtd_concat_create(struckmalloc (num_erase_region *sizeof (struct mtd_erase_region_info ),GFP_KERNEL );if (!erase_region_p) { -kfree (concat);printk ("memory allocation error while creating erase region list" " for device \"%s\"\n" , name); -return NULL ; + goto error; }/*
@@ -968,6 +968,12 @@ struct mtd_info *mtd_concat_create(struc }return &concat->mtd; + +error: +if (concat->mtd.backing_dev_info == &concat->backing_dev_info) +bdi_destroy (&concat->backing_dev_info); +kfree (concat); +return NULL ; }/*
@@ -977,6 +983,8 @@ struct mtd_info *mtd_concat_create(strucvoid mtd_concat_destroy (struct mtd_info *mtd) {struct mtd_concat *concat =CONCAT (mtd); +if (concat->mtd.backing_dev_info == &concat->backing_dev_info) +bdi_destroy (&concat->backing_dev_info);if (concat->mtd.numeraseregions)kfree (concat->mtd.eraseregions);kfree (concat);
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 16/23 mm: scalable bdi statistics counters.
In reply to Peter ZijlstraProvide scalable per backing_dev_info statistics counters.
--- include/linux/backing-dev.h | 85 ++++++++++++++++++++++++++++++++++++++++++-- mm/backing-dev.c | 27 +++++++++++++ 2 files changed, 109 insertions(+), 3 deletions(-)
include/linux/backing-dev.h +82 -3
@@ -8,6 +8,8 @@#ifndef _LINUX_BACKING_DEV_H#define _LINUX_BACKING_DEV_H +#include <linux/percpu_counter.h> +#include <linux/log2.h> #include <asm/atomic.h> struct page ;
@@ -24,6 +26,12 @@ enum bdi_state {typedef int (congested_fn )(void *,int ); +enum bdi_stat_item { +NR_BDI_STAT_ITEMS +}; + +#define BDI_STAT_BATCH (8*(1+ilog2(nr_cpu_ids))) +struct backing_dev_info {unsigned long ra_pages;/* max readahead in PAGE_CACHE_SIZE units */ unsigned long state;/* Always use atomic bitops on this */
@@ -32,15 +40,86 @@ struct backing_dev_info {void *congested_data;/* Pointer to aux data for congested func */ void (*unplug_io_fn)(struct backing_dev_info *,struct page *);void *unplug_io_data; + +struct percpu_counter bdi_stat[NR_BDI_STAT_ITEMS ]; }; -static inline int bdi_init (struct backing_dev_info *bdi) +int bdi_init (struct backing_dev_info *bdi); +void bdi_destroy (struct backing_dev_info *bdi); + +static inline void __add_bdi_stat (struct backing_dev_info *bdi, +enum bdi_stat_item item,s64 amount) { -return 0 ; +__percpu_counter_add (&bdi->bdi_stat[item], amount,BDI_STAT_BATCH ); } -static inline void bdi_destroy (struct backing_dev_info *bdi) +static inline void __inc_bdi_stat (struct backing_dev_info *bdi, +enum bdi_stat_item item) { +__add_bdi_stat (bdi, item,1 ); +} + +static inline void inc_bdi_stat (struct backing_dev_info *bdi, +enum bdi_stat_item item) +{ +unsigned long flags; + +local_irq_save (flags); +__inc_bdi_stat (bdi, item); +local_irq_restore (flags); +} + +static inline void __dec_bdi_stat (struct backing_dev_info *bdi, +enum bdi_stat_item item) +{ +__add_bdi_stat (bdi, item,-1 ); +} + +static inline void dec_bdi_stat (struct backing_dev_info *bdi, +enum bdi_stat_item item) +{ +unsigned long flags; + +local_irq_save (flags); +__dec_bdi_stat (bdi, item); +local_irq_restore (flags); +} + +static inline s64 bdi_stat (struct backing_dev_info *bdi, +enum bdi_stat_item item) +{ +return percpu_counter_read_positive (&bdi->bdi_stat[item]); +} + +static inline s64 __bdi_stat_sum (struct backing_dev_info *bdi, +enum bdi_stat_item item) +{ +return percpu_counter_sum_positive (&bdi->bdi_stat[item]); +} + +static inline s64 bdi_stat_sum (struct backing_dev_info *bdi, +enum bdi_stat_item item) +{ +s64 sum; +unsigned long flags; + +local_irq_save (flags); + sum =__bdi_stat_sum (bdi, item); +local_irq_restore (flags); + +return sum; +} + +/* +* maximal error of a stat counter. +*/ +static inline unsigned long bdi_stat_error (struct backing_dev_info *bdi) +{ +#ifdef CONFIG_SMP +return nr_cpu_ids *BDI_STAT_BATCH ; +#else +return 1 ; +#endif }/*
mm/backing-dev.c +27 -0
@@ -5,6 +5,33 @@#include <linux/sched.h> #include <linux/module.h> +int bdi_init (struct backing_dev_info *bdi) +{ +int i, j; +int err; + +for (i =0 ; i <NR_BDI_STAT_ITEMS ; i++) { + err =percpu_counter_init_irq (&bdi->bdi_stat[i],0 ); +if (err) { +for (j =0 ; j < i; j++) +percpu_counter_destroy (&bdi->bdi_stat[i]); +break ; + } + } + +return err; +} +EXPORT_SYMBOL (bdi_init); + +void bdi_destroy (struct backing_dev_info *bdi) +{ +int i; + +for (i =0 ; i <NR_BDI_STAT_ITEMS ; i++) +percpu_counter_destroy (&bdi->bdi_stat[i]); +} +EXPORT_SYMBOL (bdi_destroy); +static wait_queue_head_t congestion_wqh[2 ] = {__WAIT_QUEUE_HEAD_INITIALIZER (congestion_wqh[0 ]),__WAIT_QUEUE_HEAD_INITIALIZER (congestion_wqh[1 ])
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 17/23 mm: count reclaimable pages per BDI
In reply to Peter ZijlstraCount per BDI reclaimable pages; nr_reclaimable = nr_dirty + nr_unstable.
--- fs/buffer.c | 2 ++ fs/nfs/write.c | 7 +++++++ include/linux/backing-dev.h | 1 + mm/page-writeback.c | 4 ++++ mm/truncate.c | 2 ++ 5 files changed, 16 insertions(+)
fs/buffer.c +2 -0
@@ -697,6 +697,8 @@ static int __set_page_dirty(struct pageif (mapping_cap_account_dirty (mapping)) {__inc_zone_page_state (page,NR_FILE_DIRTY ); +__inc_bdi_stat (mapping->backing_dev_info, +BDI_RECLAIMABLE );task_io_account_write (PAGE_CACHE_SIZE ); } radix_tree_tag_set(&mapping->page_tree,
mm/page-writeback.c +4 -0
@@ -827,6 +827,8 @@ int __set_page_dirty_nobuffers(struct paWARN_ON_ONCE (!PagePrivate (page) && !PageUptodate (page));if (mapping_cap_account_dirty (mapping)) {__inc_zone_page_state (page,NR_FILE_DIRTY ); +__inc_bdi_stat (mapping->backing_dev_info, +BDI_RECLAIMABLE );task_io_account_write (PAGE_CACHE_SIZE ); } radix_tree_tag_set(&mapping->page_tree,
@@ -961,6 +963,8 @@ int clear_page_dirty_for_io(struct page*/ if (TestClearPageDirty (page)) {dec_zone_page_state (page,NR_FILE_DIRTY ); +dec_bdi_stat (mapping->backing_dev_info, +BDI_RECLAIMABLE );return 1 ; }return 0 ;
mm/truncate.c +2 -0
@@ -72,6 +72,8 @@ void cancel_dirty_page(struct page *pagestruct address_space *mapping = page->mapping;if (mapping &&mapping_cap_account_dirty (mapping)) {dec_zone_page_state (page,NR_FILE_DIRTY ); +dec_bdi_stat (mapping->backing_dev_info, +BDI_RECLAIMABLE );if (account_size)task_io_account_cancelled_write (account_size); }
fs/nfs/write.c +7 -0
@@ -464,6 +464,7 @@ nfs_mark_request_commit(struct nfs_page NFS_PAGE_TAG_COMMIT);spin_unlock (&inode->i_lock);inc_zone_page_state (req->wb_page,NR_UNSTABLE_NFS ); +inc_bdi_stat (req->wb_page->mapping->backing_dev_info,BDI_RECLAIMABLE );__mark_inode_dirty (inode,I_DIRTY_DATASYNC ); }
@@ -550,6 +551,8 @@ static void nfs_cancel_commit_list(strucwhile (!list_empty (head)) { req =nfs_list_entry (head->next);dec_zone_page_state (req->wb_page,NR_UNSTABLE_NFS ); +dec_bdi_stat (req->wb_page->mapping->backing_dev_info, +BDI_RECLAIMABLE );nfs_list_remove_request (req);clear_bit (PG_NEED_COMMIT , &(req)->wb_flags);nfs_inode_remove_request (req);
@@ -1210,6 +1213,8 @@ nfs_commit_list(struct inode *inode, strnfs_list_remove_request (req);nfs_mark_request_commit (req);dec_zone_page_state (req->wb_page,NR_UNSTABLE_NFS ); +dec_bdi_stat (req->wb_page->mapping->backing_dev_info, +BDI_RECLAIMABLE );nfs_clear_page_tag_locked (req); }return -ENOMEM ;
@@ -1235,6 +1240,8 @@ static void nfs_commit_done(struct rpc_tnfs_list_remove_request (req);clear_bit (PG_NEED_COMMIT , &(req)->wb_flags);dec_zone_page_state (req->wb_page,NR_UNSTABLE_NFS ); +dec_bdi_stat (req->wb_page->mapping->backing_dev_info, +BDI_RECLAIMABLE ); dprintk("NFS: commit (%s/%Ld %d@%Ld)", req->wb_context->path.dentry->d_inode->i_sb->s_id,
include/linux/backing-dev.h +1 -0
@@ -27,6 +27,7 @@ enum bdi_state {typedef int (congested_fn )(void *,int );enum bdi_stat_item { +BDI_RECLAIMABLE ,NR_BDI_STAT_ITEMS };
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 18/23 mm: count writeback pages per BDI
In reply to Peter ZijlstraCount per BDI writeback pages.
--- include/linux/backing-dev.h | 1 + mm/page-writeback.c | 12 ++++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-)
mm/page-writeback.c +10 -2
@@ -981,14 +981,18 @@ int test_clear_page_writeback(struct pagint ret;if (mapping) { +struct backing_dev_info *bdi = mapping->backing_dev_info;unsigned long flags;write_lock_irqsave (&mapping->tree_lock, flags); ret =TestClearPageWriteback (page); -if (ret) +if (ret) {radix_tree_tag_clear (&mapping->page_tree,page_index (page),PAGECACHE_TAG_WRITEBACK ); +if (bdi_cap_writeback_dirty (bdi)) +__dec_bdi_stat (bdi,BDI_WRITEBACK ); + }write_unlock_irqrestore (&mapping->tree_lock, flags); }else { ret =TestClearPageWriteback (page);
@@ -1004,14 +1008,18 @@ int test_set_page_writeback(struct pageint ret;if (mapping) { +struct backing_dev_info *bdi = mapping->backing_dev_info;unsigned long flags;write_lock_irqsave (&mapping->tree_lock, flags); ret =TestSetPageWriteback (page); -if (!ret) +if (!ret) {radix_tree_tag_set (&mapping->page_tree,page_index (page),PAGECACHE_TAG_WRITEBACK ); +if (bdi_cap_writeback_dirty (bdi)) +__inc_bdi_stat (bdi,BDI_WRITEBACK ); + }if (!PageDirty (page)) radix_tree_tag_clear(&mapping->page_tree, page_index(page),
include/linux/backing-dev.h +1 -0
@@ -28,6 +28,7 @@ typedef int (congested_fn)(void *, int);enum bdi_stat_item {BDI_RECLAIMABLE , +BDI_WRITEBACK ,NR_BDI_STAT_ITEMS };
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 19/23 mm: expose BDI statistics in sysfs.
In reply to Peter ZijlstraExpose the per BDI stats in /sys/block/<dev>/queue/*
--- block/ll_rw_blk.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
block/ll_rw_blk.c +30 -0
@@ -4098,6 +4098,24 @@ static ssize_t queue_max_segments_show(sreturn queue_var_show (q->max_phys_segments, page); } +static ssize_t queue_nr_reclaimable_show (struct request_queue *q,char *page) +{ +unsigned long long nr_reclaimable = +bdi_stat (&q->backing_dev_info,BDI_RECLAIMABLE ); + +return sprintf (page,"%llu\n" , + nr_reclaimable >> (PAGE_CACHE_SHIFT -10 )); +} + +static ssize_t queue_nr_writeback_show (struct request_queue *q,char *page) +{ +unsigned long long nr_writeback = +bdi_stat (&q->backing_dev_info,BDI_WRITEBACK ); + +return sprintf (page,"%llu\n" , + nr_writeback >> (PAGE_CACHE_SHIFT -10 )); +} +static ssize_t queue_max_segments_store (struct request_queue *q,const char *page,size_t count) {
@@ -4139,6 +4157,16 @@ static struct queue_sysfs_entry queue_ma .store = queue_max_segments_store, }; +static struct queue_sysfs_entry queue_reclaimable_entry = { + .attr = {.name ="reclaimable_kb" , .mode =S_IRUGO }, + .show = queue_nr_reclaimable_show, +}; + +static struct queue_sysfs_entry queue_writeback_entry = { + .attr = {.name ="writeback_kb" , .mode =S_IRUGO }, + .show = queue_nr_writeback_show, +}; +static struct queue_sysfs_entry queue_iosched_entry = { .attr = {.name ="scheduler" , .mode =S_IRUGO |S_IWUSR }, .show = elv_iosched_show,
@@ -4151,6 +4179,8 @@ static struct attribute *default_attrs[] &queue_max_hw_sectors_entry.attr, &queue_max_sectors_entry.attr, &queue_max_segments_entry.attr, + &queue_reclaimable_entry.attr, + &queue_writeback_entry.attr, &queue_iosched_entry.attr, NULL, };
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 20/23 lib: floating proportions
In reply to Peter ZijlstraGiven a set of objects, floating proportions aims to efficiently give the proportional 'activity' of a single item as compared to the whole set. Where 'activity' is a measure of a temporal property of the items.
It is efficient in that it need not inspect any other items of the set in order to provide the answer. It is not even needed to know how many other items there are.
It has one parameter, and that is the period of 'time' over which the 'activity' is measured.
---
Changes since -v8:
- merged _single into this patch
- major cleanup
- removed the overloading of the prop_local methods
- removed the prop_adjust_shift macro [akpm]
- simplified the _single code
- limited the shift argument
- provided prop_inc_{percpu,single}
- static initializer for prop_local_single
include/linux/proportions.h | 119 +++++++++++++ lib/Makefile | 3 lib/proportions.c | 384 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 505 insertions(+), 1 deletion(-)
lib/proportions.c new file +384 -0
@@ -0,0 +1,384 @@ +/* +* Floating proportions +* +* Copyright (C) 2007 Red Hat, Inc., Peter Zijlstra <[email protected]> +* +* Description: +* +* The floating proportion is a time derivative with an exponentially decaying +* history: +* +* p_{j} = \Sum_{i=0} (dx_{j}/dt_{-i}) / 2^(1+i) +* +* Where j is an element from {prop_local}, x_{j} is j's number of events, +* and i the time period over which the differential is taken. So d/dt_{-i} is +* the differential over the i-th last period. +* +* The decaying history gives smooth transitions. The time differential carries +* the notion of speed. +* +* The denominator is 2^(1+i) because we want the series to be normalised, ie. +* +* \Sum_{i=0} 1/2^(1+i) = 1 +* +* Further more, if we measure time (t) in the same events as x; so that: +* +* t = \Sum_{j} x_{j} +* +* we get that: +* +* \Sum_{j} p_{j} = 1 +* +* Writing this in an iterative fashion we get (dropping the 'd's): +* +* if (++x_{j}, ++t > period) +* t /= 2; +* for_each (j) +* x_{j} /= 2; +* +* so that: +* +* p_{j} = x_{j} / t; +* +* We optimize away the '/= 2' for the global time delta by noting that: +* +* if (++t > period) t /= 2: +* +* Can be approximated by: +* +* period/2 + (++t % period/2) +* +* [ Furthermore, when we choose period to be 2^n it can be written in terms of +* binary operations and wraparound artefacts disappear. ] +* +* Also note that this yields a natural counter of the elapsed periods: +* +* c = t / (period/2) +* +* [ Its monotonic increasing property can be applied to mitigate the wrap- +* around issue. ] +* +* This allows us to do away with the loop over all prop_locals on each period +* expiration. By remembering the period count under which it was last accessed +* as c_{j}, we can obtain the number of 'missed' cycles from: +* +* c - c_{j} +* +* We can then lazily catch up to the global period count every time we are +* going to use x_{j}, by doing: +* +* x_{j} /= 2^(c - c_{j}), c_{j} = c +*/ + +#include <linux/proportions.h> +#include <linux/rcupdate.h> + +/* +* Limit the time part in order to ensure there are some bits left for the +* cycle counter. +*/ +#define PROP_MAX_SHIFT (3*BITS_PER_LONG/4) + +int prop_descriptor_init (struct prop_descriptor *pd,int shift) +{ +int err; + +if (shift >PROP_MAX_SHIFT ) + shift =PROP_MAX_SHIFT ; + + pd->index =0 ; + pd->pg[0 ].shift = shift; +mutex_init (&pd->mutex); + err =percpu_counter_init_irq (&pd->pg[0 ].events,0 ); +if (err) + goto out; + + err =percpu_counter_init_irq (&pd->pg[1 ].events,0 ); +if (err) +percpu_counter_destroy (&pd->pg[0 ].events); + +out: +return err; +} + +/* +* We have two copies, and flip between them to make it seem like an atomic +* update. The update is not really atomic wrt the events counter, but +* it is internally consistent with the bit layout depending on shift. +* +* We copy the events count, move the bits around and flip the index. +*/ +void prop_change_shift (struct prop_descriptor *pd,int shift) +{ +int index; +int offset; +u64 events; +unsigned long flags; + +if (shift >PROP_MAX_SHIFT ) + shift =PROP_MAX_SHIFT ; + +mutex_lock (&pd->mutex); + + index = pd->index ^1 ; + offset = pd->pg[pd->index].shift - shift; +if (!offset) + goto out; + + pd->pg[index].shift = shift; + +local_irq_save (flags); + events =percpu_counter_sum (&pd->pg[pd->index].events); +if (offset <0 ) + events <<= -offset; +else + events >>= offset; +percpu_counter_set (&pd->pg[index].events, events); + +/* +* ensure the new pg is fully written before the switch +*/ +smp_wmb (); + pd->index = index; +local_irq_restore (flags); + +synchronize_rcu (); + +out: +mutex_unlock (&pd->mutex); +} + +/* +* wrap the access to the data in an rcu_read_lock() section; +* this is used to track the active references. +*/ +static struct prop_global *prop_get_global (struct prop_descriptor *pd) +{ +int index; + +rcu_read_lock (); + index = pd->index; +/* +* match the wmb from vcd_flip() +*/ +smp_rmb (); +return &pd->pg[index]; +} + +static void prop_put_global (struct prop_descriptor *pd,struct prop_global *pg) +{ +rcu_read_unlock (); +} + +static void +prop_adjust_shift (int *pl_shift,unsigned long *pl_period,int new_shift) +{ +int offset = *pl_shift - new_shift; + +if (!offset) +return ; + +if (offset <0 ) + *pl_period <<= -offset; +else + *pl_period >>= offset; + + *pl_shift = new_shift; +} + +/* +* PERCPU +*/ + +int prop_local_init_percpu (struct prop_local_percpu *pl) +{ +spin_lock_init (&pl->lock); + pl->shift =0 ; + pl->period =0 ; +return percpu_counter_init_irq (&pl->events,0 ); +} + +void prop_local_destroy_percpu (struct prop_local_percpu *pl) +{ +percpu_counter_destroy (&pl->events); +} + +/* +* Catch up with missed period expirations. +* +* until (c_{j} == c) +* x_{j} -= x_{j}/2; +* c_{j}++; +*/ +static +void prop_norm_percpu (struct prop_global *pg,struct prop_local_percpu *pl) +{ +unsigned long period =1UL << (pg->shift -1 ); +unsigned long period_mask = ~(period -1 ); +unsigned long global_period; +unsigned long flags; + + global_period =percpu_counter_read (&pg->events); + global_period &= period_mask; + +/* +* Fast path - check if the local and global period count still match +* outside of the lock. +*/ +if (pl->period == global_period) +return ; + +spin_lock_irqsave (&pl->lock, flags); +prop_adjust_shift (&pl->shift, &pl->period, pg->shift); +/* +* For each missed period, we half the local counter. +* basically: +* pl->events >> (global_period - pl->period); +* +* but since the distributed nature of percpu counters make division +* rather hard, use a regular subtraction loop. This is safe, because +* the events will only every be incremented, hence the subtraction +* can never result in a negative number. +*/ +while (pl->period != global_period) { +unsigned long val =percpu_counter_read (&pl->events); +unsigned long half = (val +1 ) >>1 ; + +/* +* Half of zero won't be much less, break out. +* This limits the loop to shift iterations, even +* if we missed a million. +*/ +if (!val) +break ; + +percpu_counter_add (&pl->events, -half); + pl->period += period; + } + pl->period = global_period; +spin_unlock_irqrestore (&pl->lock, flags); +} + +/* +* ++x_{j}, ++t +*/ +void __prop_inc_percpu (struct prop_descriptor *pd,struct prop_local_percpu *pl) +{ +struct prop_global *pg =prop_get_global (pd); + +prop_norm_percpu (pg, pl); +percpu_counter_add (&pl->events,1 ); +percpu_counter_add (&pg->events,1 ); +prop_put_global (pd, pg); +} + +/* +* Obtain a fraction of this proportion +* +* p_{j} = x_{j} / (period/2 + t % period/2) +*/ +void prop_fraction_percpu (struct prop_descriptor *pd, +struct prop_local_percpu *pl, +long *numerator,long *denominator) +{ +struct prop_global *pg =prop_get_global (pd); +unsigned long period_2 =1UL << (pg->shift -1 ); +unsigned long counter_mask = period_2 -1 ; +unsigned long global_count; + +prop_norm_percpu (pg, pl); + *numerator =percpu_counter_read_positive (&pl->events); + + global_count =percpu_counter_read (&pg->events); + *denominator = period_2 + (global_count & counter_mask); + +prop_put_global (pd, pg); +} + +/* +* SINGLE +*/ + +int prop_local_init_single (struct prop_local_single *pl) +{ +spin_lock_init (&pl->lock); + pl->shift =0 ; + pl->period =0 ; + pl->events =0 ; +return 0 ; +} + +void prop_local_destroy_single (struct prop_local_single *pl) +{ +} + +/* +* Catch up with missed period expirations. +*/ +static +void prop_norm_single (struct prop_global *pg,struct prop_local_single *pl) +{ +unsigned long period =1UL << (pg->shift -1 ); +unsigned long period_mask = ~(period -1 ); +unsigned long global_period; +unsigned long flags; + + global_period =percpu_counter_read (&pg->events); + global_period &= period_mask; + +/* +* Fast path - check if the local and global period count still match +* outside of the lock. +*/ +if (pl->period == global_period) +return ; + +spin_lock_irqsave (&pl->lock, flags); +prop_adjust_shift (&pl->shift, &pl->period, pg->shift); +/* +* For each missed period, we half the local counter. +*/ + period = (global_period - pl->period) >> (pg->shift -1 ); +if (likely (period <BITS_PER_LONG )) + pl->events >>= period; +else + pl->events =0 ; + pl->period = global_period; +spin_unlock_irqrestore (&pl->lock, flags); +} + +/* +* ++x_{j}, ++t +*/ +void __prop_inc_single (struct prop_descriptor *pd,struct prop_local_single *pl) +{ +struct prop_global *pg =prop_get_global (pd); + +prop_norm_single (pg, pl); + pl->events++; +percpu_counter_add (&pg->events,1 ); +prop_put_global (pd, pg); +} + +/* +* Obtain a fraction of this proportion +* +* p_{j} = x_{j} / (period/2 + t % period/2) +*/ +void prop_fraction_single (struct prop_descriptor *pd, +struct prop_local_single *pl, +long *numerator,long *denominator) +{ +struct prop_global *pg =prop_get_global (pd); +unsigned long period_2 =1UL << (pg->shift -1 ); +unsigned long counter_mask = period_2 -1 ; +unsigned long global_count; + +prop_norm_single (pg, pl); + *numerator = pl->events; + + global_count =percpu_counter_read (&pg->events); + *denominator = period_2 + (global_count & counter_mask); + +prop_put_global (pd, pg); +}
include/linux/proportions.h new file +119 -0
119 lines not on this page · raw message
lib/Makefile +2 -1
@@ -5,7 +5,8 @@ lib-y := ctype.o string.o vsprintf.o kasprintf.o cmdline.o \ rbtree.o radix-tree.o dump_stack.o \ idr.o int_sqrt.o bitmap.o extable.o prio_tree.o \ - sha1.o irq_regs.o reciprocal_div.o argv_split.o + sha1.o irq_regs.o reciprocal_div.o argv_split.o \ + proportions.o lib-$(CONFIG_MMU) += ioremap.o pagewalk.o lib-$(CONFIG_SMP) += cpumask.o
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 21/23 mm: per device dirty threshold
In reply to Peter ZijlstraScale writeback cache per backing device, proportional to its writeout speed.
By decoupling the BDI dirty thresholds a number of problems we currently have will go away, namely:
- mutual interference starvation (for any number of BDIs);
- deadlocks with stacked BDIs (loop, FUSE and local NFS mounts).
It might be that all dirty pages are for a single BDI while other BDIs are idling. By giving each BDI a 'fair' share of the dirty limit, each one can have dirty pages outstanding and make progress.
A global threshold also creates a deadlock for stacked BDIs; when A writes to B, and A generates enough dirty pages to get throttled, B will never start writeback until the dirty pages go away. Again, by giving each BDI its own 'independent' dirty limit, this problem is avoided.
So the problem is to determine how to distribute the total dirty limit across the BDIs fairly and efficiently. A DBI that has a large dirty limit but does not have any dirty pages outstanding is a waste.
What is done is to keep a floating proportion between the DBIs based on writeback completions. This way faster/more active devices get a larger share than slower/idle devices.
---
Changes since -v8:
- explicit usage of prop_local_percpu
- moved dirty_ratio_handler declaration into a suitable header file
include/linux/backing-dev.h | 4 include/linux/writeback.h | 4 kernel/sysctl.c | 2 mm/backing-dev.c | 19 +++- mm/page-writeback.c | 202 +++++++++++++++++++++++++++++++++++++------- 5 files changed, 196 insertions(+), 35 deletions(-)
include/linux/backing-dev.h +4 -0
@@ -10,6 +10,7 @@#include <linux/percpu_counter.h> #include <linux/log2.h> +#include <linux/proportions.h> #include <asm/atomic.h> struct page ;
@@ -44,6 +45,9 @@ struct backing_dev_info {void *unplug_io_data;struct percpu_counter bdi_stat[NR_BDI_STAT_ITEMS ]; + +struct prop_local_percpu completions; +int dirty_exceeded; };int bdi_init (struct backing_dev_info *bdi);
mm/page-writeback.c +173 -29
301 lines not on this page · raw message
mm/backing-dev.c +14 -5
@@ -12,11 +12,17 @@ int bdi_init(struct backing_dev_info *bdfor (i =0 ; i <NR_BDI_STAT_ITEMS ; i++) { err =percpu_counter_init_irq (&bdi->bdi_stat[i],0 ); -if (err) { -for (j =0 ; j < i; j++) -percpu_counter_destroy (&bdi->bdi_stat[i]); -break ; - } +if (err) + goto err; + } + + bdi->dirty_exceeded =0 ; + err =prop_local_init_percpu (&bdi->completions); + +if (err) { +err: +for (j =0 ; j < i; j++) +percpu_counter_destroy (&bdi->bdi_stat[i]); }return err;
@@ -29,6 +35,8 @@ void bdi_destroy(struct backing_dev_infofor (i =0 ; i <NR_BDI_STAT_ITEMS ; i++)percpu_counter_destroy (&bdi->bdi_stat[i]); + +prop_local_destroy_percpu (&bdi->completions); }EXPORT_SYMBOL (bdi_destroy);
@@ -81,3 +89,4 @@ long congestion_wait(int rw, long timeoureturn ret; }EXPORT_SYMBOL (congestion_wait); +
kernel/sysctl.c +1 -1
@@ -827,7 +827,7 @@ static struct ctl_table vm_table[] = { .data = &vm_dirty_ratio, .maxlen = sizeof(vm_dirty_ratio), .mode = 0644, - .proc_handler = &proc_dointvec_minmax, + .proc_handler = &dirty_ratio_handler, .strategy = &sysctl_intvec, .extra1 = &zero, .extra2 = &one_hundred,
include/linux/writeback.h +4 -0
@@ -104,6 +104,10 @@ extern int dirty_expire_interval;extern int block_dump;extern int laptop_mode; +extern int dirty_ratio_handler (struct ctl_table *table,int write, +struct file *filp,void __user *buffer,size_t *lenp, +loff_t *ppos); +struct ctl_table ;struct file ; int dirty_writeback_centisecs_handler(struct ctl_table *, int, struct file *,
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 21/23] mm: per device dirty threshold
Peter> Scale writeback cache per backing device, proportional to its Peter> writeout speed. By decoupling the BDI dirty thresholds a Peter> number of problems we currently have will go away, namely:
Ah, this clarifies my questions! Thanks!
Peter> - mutual interference starvation (for any number of BDIs); Peter> - deadlocks with stacked BDIs (loop, FUSE and local NFS mounts).
Peter> It might be that all dirty pages are for a single BDI while Peter> other BDIs are idling. By giving each BDI a 'fair' share of the Peter> dirty limit, each one can have dirty pages outstanding and make Peter> progress.
Question, can you change (shrink) the limit on a BDI while it has IO in flight? And what will that do to the system? I.e. if you have one device doing IO, so that it has a majority of the dirty limit. Then another device starts IO, and it's a *faster* device, how quickly/slowly does the BDI dirty limits change for both the old and new device?
Peter> A global threshold also creates a deadlock for stacked BDIs; Peter> when A writes to B, and A generates enough dirty pages to get Peter> throttled, B will never start writeback until the dirty pages Peter> go away. Again, by giving each BDI its own 'independent' dirty Peter> limit, this problem is avoided.
Peter> So the problem is to determine how to distribute the total Peter> dirty limit across the BDIs fairly and efficiently. A DBI that
You mean BDI here, not DBI.
Peter> has a large dirty limit but does not have any dirty pages Peter> outstanding is a waste.
Peter> What is done is to keep a floating proportion between the DBIs Peter> based on writeback completions. This way faster/more active Peter> devices get a larger share than slower/idle devices.
Does a slower device get a BDI which is calculated to keep it's limit under a certain number of seconds of outstanding IO? This way no device can build up more than say 15 seconds of outstanding IO to flush at any one time.
Thanks!
John
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 21/23] mm: per device dirty threshold
Quoted 14 lines · John Stoffel
Peter> Scale writeback cache per backing device, proportional to its Peter> writeout speed. By decoupling the BDI dirty thresholds a Peter> number of problems we currently have will go away, namely:
Ah, this clarifies my questions! Thanks!
Peter> - mutual interference starvation (for any number of BDIs); Peter> - deadlocks with stacked BDIs (loop, FUSE and local NFS mounts).
Peter> It might be that all dirty pages are for a single BDI while Peter> other BDIs are idling. By giving each BDI a 'fair' share of the Peter> dirty limit, each one can have dirty pages outstanding and make Peter> progress.
Quoted 6 lines
Question, can you change (shrink) the limit on a BDI while it has IO in flight? And what will that do to the system? I.e. if you have one device doing IO, so that it has a majority of the dirty limit. Then another device starts IO, and it's a *faster* device, how quickly/slowly does the BDI dirty limits change for both the old and new device?
Yes, it can change while in use. A measure of how quickly it can change is roughly: it can half in a dirty_limit worth of writeout.
What will happen is that those processes doing heavy IO on the slower device will get throttled more aggressively until its below its new threshold again - however all the time it will keep on writing at (full) speed because it will have this backlog to rid itself of, and by doing that it completes writeouts which ensure it will keep part of the dirty limit for itself, and thus can always make progress.
You can monitor this by looking at /sys/block/sd*/queue/cache_size while doing such a thing. It should stabilise quite 'quickly'.
Quoted 10 lines
Peter> A global threshold also creates a deadlock for stacked BDIs; Peter> when A writes to B, and A generates enough dirty pages to get Peter> throttled, B will never start writeback until the dirty pages Peter> go away. Again, by giving each BDI its own 'independent' dirty Peter> limit, this problem is avoided.
Peter> So the problem is to determine how to distribute the total Peter> dirty limit across the BDIs fairly and efficiently. A DBI that
You mean BDI here, not DBI.
Uhh, yeah, obviously :-)
Quoted 11 lines
Peter> has a large dirty limit but does not have any dirty pages Peter> outstanding is a waste.
Peter> What is done is to keep a floating proportion between the DBIs Peter> based on writeback completions. This way faster/more active Peter> devices get a larger share than slower/idle devices.
Does a slower device get a BDI which is calculated to keep it's limit under a certain number of seconds of outstanding IO? This way no device can build up more than say 15 seconds of outstanding IO to flush at any one time.
Perhaps already answered above, as long as there is dirty stuff to write out it will keep completing writes and thus gain a stable share of the dirty limit.
PATCH 22/23 mm: dirty balancing for tasks
In reply to Peter ZijlstraBased on ideas of Andrew:
http://marc.info/?l=linux-kernel&m=102912915020543&w=2
Scale the bdi dirty limit inversly with the tasks dirty rate. This makes heavy writers have a lower dirty limit than the occasional writer.
Andrea proposed something similar:
http://lwn.net/Articles/152277/
The main disadvantage to his patch is that he uses an unrelated quantity to measure time, which leaves him with a workload dependant tunable. Other than that the two approaches appear quite similar.
---
Changes since -v8:
- initialized init_task
- moved the prop_local init after the task_struct copy
- changed the per task ratio to 1/8 (from 1/2).
- explicit usage of prop_local_single
include/linux/init_task.h | 1 include/linux/sched.h | 2 + kernel/fork.c | 10 +++++++++ mm/page-writeback.c | 50 +++++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 62 insertions(+), 1 deletion(-)
include/linux/sched.h +2 -0
@@ -84,6 +84,7 @@ struct sched_param {#include <linux/timer.h> #include <linux/hrtimer.h> #include <linux/task_io_accounting.h> +#include <linux/proportions.h> #include <asm/processor.h>
@@ -1125,6 +1126,7 @@ struct task_struct {#ifdef CONFIG_FAULT_INJECTION int make_it_fail;#endif +struct prop_local_single dirties; };/*
kernel/fork.c +10 -0
@@ -107,6 +107,7 @@ static struct kmem_cache *mm_cachep;void free_task (struct task_struct *tsk) { +prop_local_destroy_single (&tsk->dirties);free_thread_info (tsk->stack);rt_mutex_debug_task_free (tsk);free_task_struct (tsk);
@@ -163,6 +164,7 @@ static struct task_struct *dup_task_stru {struct task_struct *tsk;struct thread_info *ti; +int err;prepare_to_copy (orig);
@@ -178,6 +180,14 @@ static struct task_struct *dup_task_stru *tsk = *orig; tsk->stack = ti; + + err =prop_local_init_single (&tsk->dirties); +if (err) { +free_thread_info (ti); +free_task_struct (tsk); +return NULL ; + } +setup_thread_stack (tsk, orig);#ifdef CONFIG_CC_STACKPROTECTOR
mm/page-writeback.c +49 -1
98 lines not on this page · raw message
include/linux/init_task.h +1 -0
@@ -169,6 +169,7 @@ extern struct group_info init_groups; [PIDTYPE_PGID] = INIT_PID_LINK(PIDTYPE_PGID), \ [PIDTYPE_SID] = INIT_PID_LINK(PIDTYPE_SID), \ }, \ + .dirties = INIT_PROP_LOCAL_SINGLE(dirties), \ INIT_TRACE_IRQFLAGS \ INIT_LOCKDEP \ }
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 23/23 debug: sysfs files for the current ratio/size/total
In reply to Peter ZijlstraExpose the per bdi dirty limits in sysfs
--- block/ll_rw_blk.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++ mm/page-writeback.c | 4 +- 2 files changed, 83 insertions(+), 2 deletions(-)
block/ll_rw_blk.c +81 -0
@@ -4128,6 +4128,57 @@ static ssize_t queue_max_segments_store(return ret; } + +extern void bdi_writeout_fraction (struct backing_dev_info *bdi, +long *numerator,long *denominator); + +static ssize_t queue_nr_cache_ratio_show (struct request_queue *q,char *page) +{ +long scale, div; + +bdi_writeout_fraction (&q->backing_dev_info, &scale, &div); + scale *=1024 ; + scale /= div; + +return sprintf (page,"%ld\n" , scale); +} + +static ssize_t queue_nr_cache_num_show (struct request_queue *q,char *page) +{ +long scale, div; + +bdi_writeout_fraction (&q->backing_dev_info, &scale, &div); + +return sprintf (page,"%ld\n" , scale); +} + +static ssize_t queue_nr_cache_denom_show (struct request_queue *q,char *page) +{ +long scale, div; + +bdi_writeout_fraction (&q->backing_dev_info, &scale, &div); + +return sprintf (page,"%ld\n" , div); +} + +extern void +get_dirty_limits (long *pbackground,long *pdirty,long *pbdi_dirty, +struct backing_dev_info *bdi); + +static ssize_t queue_nr_cache_size_show (struct request_queue *q,char *page) +{ +long background, dirty, bdi_dirty; +get_dirty_limits (&background, &dirty, &bdi_dirty, &q->backing_dev_info); +return sprintf (page,"%ld\n" , bdi_dirty); +} + +static ssize_t queue_nr_cache_total_show (struct request_queue *q,char *page) +{ +long background, dirty, bdi_dirty; +get_dirty_limits (&background, &dirty, &bdi_dirty, &q->backing_dev_info); +return sprintf (page,"%ld\n" , dirty); +} +static struct queue_sysfs_entry queue_requests_entry = { .attr = {.name ="nr_requests" , .mode =S_IRUGO |S_IWUSR }, .show = queue_requests_show,
@@ -4167,6 +4218,31 @@ static struct queue_sysfs_entry queue_wr .show = queue_nr_writeback_show, }; +static struct queue_sysfs_entry queue_cache_ratio_entry = { + .attr = {.name ="cache_ratio" , .mode =S_IRUGO }, + .show = queue_nr_cache_ratio_show, +}; + +static struct queue_sysfs_entry queue_cache_num_entry = { + .attr = {.name ="cache_num" , .mode =S_IRUGO }, + .show = queue_nr_cache_num_show, +}; + +static struct queue_sysfs_entry queue_cache_denom_entry = { + .attr = {.name ="cache_denom" , .mode =S_IRUGO }, + .show = queue_nr_cache_denom_show, +}; + +static struct queue_sysfs_entry queue_cache_size_entry = { + .attr = {.name ="cache_size" , .mode =S_IRUGO }, + .show = queue_nr_cache_size_show, +}; + +static struct queue_sysfs_entry queue_cache_total_entry = { + .attr = {.name ="cache_total" , .mode =S_IRUGO }, + .show = queue_nr_cache_total_show, +}; +static struct queue_sysfs_entry queue_iosched_entry = { .attr = {.name ="scheduler" , .mode =S_IRUGO |S_IWUSR }, .show = elv_iosched_show,
@@ -4181,6 +4257,11 @@ static struct attribute *default_attrs[] &queue_max_segments_entry.attr, &queue_reclaimable_entry.attr, &queue_writeback_entry.attr, + &queue_cache_ratio_entry.attr, + &queue_cache_num_entry.attr, + &queue_cache_denom_entry.attr, + &queue_cache_size_entry.attr, + &queue_cache_total_entry.attr, &queue_iosched_entry.attr, NULL, };
mm/page-writeback.c +2 -2
@@ -169,7 +169,7 @@ static inline void task_dirty_inc(struct/* * Obtain an accurate fraction of the BDI's portion. */ -static void bdi_writeout_fraction (struct backing_dev_info *bdi, +void bdi_writeout_fraction (struct backing_dev_info *bdi,long *numerator,long *denominator) {if (bdi_cap_writeback_dirty (bdi)) {
@@ -291,7 +291,7 @@ static unsigned long determine_dirtyablereturn x +1 ;/* Ensure that we never return 0 */ } -static void +void get_dirty_limits (long *pbackground,long *pdirty,long *pbdi_dirty,struct backing_dev_info *bdi) {
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Peter> Per device dirty throttling patches These patches aim to Peter> improve balance_dirty_pages() and directly address three Peter> issues:
Peter> 1) inter device starvation Peter> 2) stacked device deadlocks Peter> 3) inter process starvation
Peter> 1 and 2 are a direct result from removing the global dirty Peter> limit and using per device dirty limits. By giving each device Peter> its own dirty limit is will no longer starve another device, Peter> and the cyclic dependancy on the dirty limit is broken.
Ye haa! This should be a big improvement.
Peter> In order to efficiently distribute the dirty limit across the Peter> independant devices a floating proportion is used, this will Peter> allocate a share of the total limit proportional to the Peter> device's recent activity.
I'm not sure I like or agree with this. Shouldn't we be limiting based on the device's capability to sustain traffic? So if I have a RAID device which can read/write a total of 100Mb/sec, while at the same time I've got a CF device which can do 5Mb/sec, shouldn't we be more strongly limiting the CF device, even if it is the only device being written to?
Of course, I haven't read the patches yet, nor am I qualified to comment on them in any meanginful way I think. Hopefully I'm just missing something key here in the explanation.
Peter> 3 is done by also scaling the dirty limit proportional to the Peter> current task's recent dirty rate.
Do you mean task or device here? I'm just wondering how well this works with a bunch of devices with wildly varying speeds.
John
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
On Tue, 2007-09-11 at 22:31 -0400, John Stoffel wrote:
I hope the snipped questions were sufficiently answered in the other mail. If not, holler :-)
Quoted 5 lines
Peter> 3 is done by also scaling the dirty limit proportional to the Peter> current task's recent dirty rate.
Do you mean task or device here? I'm just wondering how well this works with a bunch of devices with wildly varying speeds.
Task. What I do is modify the limit like this:
current_limit = dirty_limit * p(bdi_writeout) * (1 - p(task_dirty)/8)
Where the p() values are [0, 1].
By including the inverse of the task dirty rate one gets that tasks that are more agressive dirtiers get throttled more aggressively, whereas tasks that occasionally dirty a page get a little more room.