dquot: do full inode dirty in allocating space
PATCH 1/1 dquot: do full inode dirty in allocating space
Alex Shi found a regression when doing ffsb test. The test has several threads, and each thread creates a small file, write to it and then delete it. ffsb reports about 20% regression and Alex bisected it to 43d2932d88e4. The test will call __mark_inode_dirty 3 times. without this commit, we only take inode_lock one time, while with it, we take the lock 3 times with flags ( I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased too much. Below proposed patch fixes it.
fs is allocating blocks, which usually means file writes and the inode will be dirtied soon. We fully dirty the inode to reduce some inode_lock contention in several calls of __mark_inode_dirty.
include/linux/quotaops.h +1 -1
@@ -275,7 +275,7 @@ static inline int dquot_alloc_space(struct inode *inode, qsize_t nr) ret =dquot_alloc_space_nodirty (inode, nr);if (!ret) -mark_inode_dirty_sync (inode); +mark_inode_dirty (inode);return ret; }
Hi,
Quoted 7 lines · Shaohua Li
Alex Shi found a regression when doing ffsb test. The test has several threads, and each thread creates a small file, write to it and then delete it. ffsb reports about 20% regression and Alex bisected it to
43d2932d88e4. The test will call__mark_inode_dirty3 times. without this commit, we only takeinode_lockone time, while with it, we take the lock 3 times with flags ( I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased too much. Below proposed patch fixes it.
Thanks for the analysis and the patch! With which filesystem have you
measured the results? And what kind of machine it was?
Quoted 3 lines
fs is allocating blocks, which usually means file writes and the inode will be dirtied soon. We fully dirty the inode to reduce some
inode_lockcontention in several calls of__mark_inode_dirty.
Well, this is rather a workaround for a poor handling of inode dirty
bits. BTW, I think Nick's VFS scalability patches address inode_lock contention as well so with them the contention you see should be reduced.
Anyway, I will take this patch for the time before inode_lock
contention improves and add a proper comment about this before mark_inode_dirty.
Honza
Quoted 17 lines
Signed-off-by Shaohua Li <[email protected]>Signed-off-by Alex Shi <[email protected]>include/linux/quotaops.h +1
-1@@ -275,7 +275,7 @@ static inline int dquot_alloc_space(struct inode *inode, qsize_t nr) ret =dquot_alloc_space_nodirty (inode, nr);if (!ret) -mark_inode_dirty_sync (inode); +mark_inode_dirty (inode);return ret; }
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 16 lines · Jan Kara
Hi,
Quoted 7 lines · Shaohua Li
Alex Shi found a regression when doing ffsb test. The test has several threads, and each thread creates a small file, write to it and then delete it. ffsb reports about 20% regression and Alex bisected it to
43d2932d88e4. The test will call__mark_inode_dirty3 times. without this commit, we only takeinode_lockone time, while with it, we take the lock 3 times with flags ( I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased too much. Below proposed patch fixes it.Thanks for the analysis and the patch! With which filesystem have you
measured the results? And what kind of machine it was?Quoted 3 lines
fs is allocating blocks, which usually means file writes and the inode will be dirtied soon. We fully dirty the inode to reduce some
inode_lockcontention in several calls of__mark_inode_dirty.
Quoted 6 lines
Well, this is rather a workaround for a poor handling of inode dirty
bits. BTW, I think Nick's VFS scalability patches addressinode_lockcontention as well so with them the contention you see should be reduced.
Anyway, I will take this patch for the time beforeinode_lock
contention improves and add a proper comment about this beforemark_inode_dirty.
Attached is a version of the patch that is in my tree now.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 1 line
From
2a7ad4df051b43a481862fe0afa50539bc7e6b78Mon Sep 17 00:00:00 2001
From: Shaohua Li <[email protected]>
Date: Fri, 20 Aug 2010 16:49:43 +0800
Subject: [PATCH] dquot: do full inode dirty in allocating space
Alex Shi found a regression when doing ffsb test. The test has several threads, and each thread creates a small file, write to it and then delete it. ffsb reports about 20% regression and Alex bisected it to 43d2932d88e4. The test will call __mark_inode_dirty 3 times. without this commit, we only take inode_lock one time, while with it, we take the lock 3 times with flags ( I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased too much. Below proposed patch fixes it.
fs is allocating blocks, which usually means file writes and the inode will be dirtied soon. We fully dirty the inode to reduce some inode_lock contention in several calls of __mark_inode_dirty.
Jan Kara: Added comment.
--- include/linux/quotaops.h | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-)
include/linux/quotaops.h +8 -2
@@ -274,8 +274,14 @@ static inline int dquot_alloc_space(struct inode *inode, qsize_t nr)int ret; ret =dquot_alloc_space_nodirty (inode, nr); -if (!ret) -mark_inode_dirty_sync (inode); +if (!ret) { +/* +* Mark inode fully dirty. Since we are allocating blocks, inode +* would become fully dirty soon anyway and it reportedly +* reduces inode_lock contention. +*/ +mark_inode_dirty (inode); + }return ret; }
Signature · 1 line
1.6.4.2
Quoted 10 lines · Jan Kara
Hi,
Quoted 7 lines · Shaohua Li
Alex Shi found a regression when doing ffsb test. The test has several threads, and each thread creates a small file, write to it and then delete it. ffsb reports about 20% regression and Alex bisected it to
43d2932d88e4. The test will call__mark_inode_dirty3 times. without this commit, we only takeinode_lockone time, while with it, we take the lock 3 times with flags ( I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased too much. Below proposed patch fixes it.
Quoted 2 lines
Thanks for the analysis and the patch! With which filesystem have you
measured the results? And what kind of machine it was?
it's ext3 and 24 CPU system with two sockets.
Quoted 6 lines
Quoted 3 lines
fs is allocating blocks, which usually means file writes and the inode will be dirtied soon. We fully dirty the inode to reduce some
inode_lockcontention in several calls of__mark_inode_dirty.Well, this is rather a workaround for a poor handling of inode dirty
bits. BTW, I think Nick's VFS scalability patches addressinode_lockcontention as well so with them the contention you see should be reduced.
yep, the VFS scalability patch might reduce this.
Quoted 3 lines
Anyway, I will take this patch for the time before
inode_lock
contention improves and add a proper comment about this beforemark_inode_dirty.
Thanks.