Quota scalability patches
Subsystems (4)
Jan Kara
reviewerv2 cover Quota scalability patches
Hello,
this is a series of patches that address scalability issues in quota subsystem. For my testing I've used a synthetic benchmark where 50 different processes are creating files 10000 files each in separate directories (either for different or the same user). I've been testing on a machine with 48 CPUs with 64GB of RAM over either SATA drive or ramdisk. Different patches have shown improvements in different configs (see details in the changelog). Also Wang Shilong gave some testing to my patches and in his setup he saw about 100% improvement for file creation and about 50% improvement for file unlink.
The patches have passed xfstests on ext4.
First 14 patches convert users of a global dqio_mutex to rw semaphore and also avoid using the semaphore altogether in cases where we don't need it - in particular we now use dquot->dq_lock for protecting writeout of dquot (which happens very frequently on every usage change). Patches 15-18 are smaller fixes / cleanups. Patches 19-22 allow filesystem to not use dirty dquot list (as e.g. ext4 does not need it), reducing contention on global dq_list_lock significantly. Patches 23-27 remove use of global dq_data_lock for quota usage changes (by using new dquot->dq_dqb_lock).
Full series can be also fetched from my tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git quota_scaling
Unless someone objects, I'll queue the patches in my tree for the next merge window.
Changes since v1:
* Improvements from Andreas Dilger
* Added Andreas' Reviewed-by tags
Honza
Jan Kara
reviewerPATCH v2 1/27 quota: Convert dqio_mutex to rwsem
Convert dqio_mutex to rwsem and call it dqio_sem. No functional changes yet.
--- fs/ext4/super.c | 13 ++++--------- fs/ocfs2/quota_global.c | 20 ++++++++++---------- fs/ocfs2/quota_local.c | 10 +++++----- fs/quota/dquot.c | 28 ++++++++++++++-------------- fs/quota/quota_tree.c | 2 +- fs/super.c | 2 +- include/linux/quota.h | 2 +- 7 files changed, 36 insertions(+), 41 deletions(-)
fs/ext4/super.c +4 -9
@@ -5263,18 +5263,13 @@ static int ext4_statfs(struct dentry *dentry, struct kstatfs *buf)return 0 ; } -/* Helper function for writing quotas on sync - we need to start transaction -* before quota file is locked for write. Otherwise the are possible deadlocks: -* Process 1 Process 2 -* ext4_create() quota_sync() -* jbd2_journal_start() write_dquot() -* dquot_initialize() down(dqio_mutex) -* down(dqio_mutex) jbd2_journal_start() -* -*/ #ifdef CONFIG_QUOTA +/* +* Helper functions so that transaction is started before we acquire dqio_sem +* to keep correct lock ordering of transaction > dqio_sem +*/ static inline struct inode *dquot_to_inode (struct dquot *dquot) {return sb_dqopt (dquot->dq_sb)->files[dquot->dq_id.type];
fs/ocfs2/quota_global.c +10 -10
@@ -33,7 +33,7 @@* Locking of quotas with OCFS2 is rather complex. Here are rules that * should be obeyed by all the functions: * - any write of quota structure (either to local or global file) is protected -* by dqio_mutex or dquot->dq_lock. +* by dqio_sem or dquot->dq_lock. * - any modification of global quota file holds inode cluster lock, i_mutex, * and ip_alloc_sem of the global quota file (achieved by * ocfs2_lock_global_qf). It also has to hold qinfo_lock.
@@ -42,9 +42,9 @@* * A rough sketch of locking dependencies (lf = local file, gf = global file): * Normal filesystem operation: -* start_trans -> dqio_mutex -> write to lf +* start_trans -> dqio_sem -> write to lf * Syncing of local and global file: -* ocfs2_lock_global_qf -> start_trans -> dqio_mutex -> qinfo_lock -> +* ocfs2_lock_global_qf -> start_trans -> dqio_sem -> qinfo_lock -> * write to gf * -> write to lf * Acquire dquot for the first time:
@@ -60,7 +60,7 @@* Recovery: * inode cluster lock of recovered lf * -> read bitmaps -> ip_alloc_sem of lf -* -> ocfs2_lock_global_qf -> start_trans -> dqio_mutex -> qinfo_lock -> +* -> ocfs2_lock_global_qf -> start_trans -> dqio_sem -> qinfo_lock -> * write to gf */
@@ -611,7 +611,7 @@ static int ocfs2_sync_dquot_helper(struct dquot *dquot, unsigned long type)mlog_errno (status); goto out_ilock; } -mutex_lock (&sb_dqopt (sb)->dqio_mutex); +down_write (&sb_dqopt (sb)->dqio_sem); status =ocfs2_sync_dquot (dquot);if (status <0 )mlog_errno (status);
@@ -619,7 +619,7 @@ static int ocfs2_sync_dquot_helper(struct dquot *dquot, unsigned long type) status =ocfs2_local_write_dquot (dquot);if (status <0 )mlog_errno (status); -mutex_unlock (&sb_dqopt (sb)->dqio_mutex); +up_write (&sb_dqopt (sb)->dqio_sem);ocfs2_commit_trans (osb, handle); out_ilock:ocfs2_unlock_global_qf (oinfo,1 );
@@ -666,9 +666,9 @@ static int ocfs2_write_dquot(struct dquot *dquot)mlog_errno (status); goto out; } -mutex_lock (&sb_dqopt (dquot->dq_sb)->dqio_mutex); +down_write (&sb_dqopt (dquot->dq_sb)->dqio_sem); status =ocfs2_local_write_dquot (dquot); -mutex_unlock (&sb_dqopt (dquot->dq_sb)->dqio_mutex); +up_write (&sb_dqopt (dquot->dq_sb)->dqio_sem);ocfs2_commit_trans (osb, handle); out:return status;
@@ -939,7 +939,7 @@ static int ocfs2_mark_dquot_dirty(struct dquot *dquot)mlog_errno (status); goto out_ilock; } -mutex_lock (&sb_dqopt (sb)->dqio_mutex); +down_write (&sb_dqopt (sb)->dqio_sem); status =ocfs2_sync_dquot (dquot);if (status <0 ) {mlog_errno (status);
@@ -948,7 +948,7 @@ static int ocfs2_mark_dquot_dirty(struct dquot *dquot)/* Now write updated local dquot structure */ status =ocfs2_local_write_dquot (dquot); out_dlock: -mutex_unlock (&sb_dqopt (sb)->dqio_mutex); +up_write (&sb_dqopt (sb)->dqio_sem);ocfs2_commit_trans (osb, handle); out_ilock:ocfs2_unlock_global_qf (oinfo,1 );
fs/ocfs2/quota_local.c +5 -5
@@ -520,7 +520,7 @@ static int ocfs2_recover_local_quota_file(struct inode *lqinode,mlog_errno (status); goto out_drop_lock; } -mutex_lock (&sb_dqopt (sb)->dqio_mutex); +down_write (&sb_dqopt (sb)->dqio_sem);spin_lock (&dq_data_lock);/* Add usage from quota entry into quota changes * of our node. Auxiliary variables are important
@@ -553,7 +553,7 @@ static int ocfs2_recover_local_quota_file(struct inode *lqinode,unlock_buffer (qbh);ocfs2_journal_dirty (handle, qbh); out_commit: -mutex_unlock (&sb_dqopt (sb)->dqio_mutex); +up_write (&sb_dqopt (sb)->dqio_sem);ocfs2_commit_trans (OCFS2_SB (sb), handle); out_drop_lock:ocfs2_unlock_global_qf (oinfo,1 );
@@ -693,7 +693,7 @@ static int ocfs2_local_read_info(struct super_block *sb, int type)/* We don't need the lock and we have to acquire quota file locks * which will later depend on this lock */ -mutex_unlock (&sb_dqopt (sb)->dqio_mutex); +up_write (&sb_dqopt (sb)->dqio_sem); info->dqi_max_spc_limit =0x7fffffffffffffffLL ; info->dqi_max_ino_limit =0x7fffffffffffffffLL ; oinfo =kmalloc (sizeof (struct ocfs2_mem_dqinfo ),GFP_NOFS );
@@ -772,7 +772,7 @@ static int ocfs2_local_read_info(struct super_block *sb, int type) goto out_err; } -mutex_lock (&sb_dqopt (sb)->dqio_mutex); +down_write (&sb_dqopt (sb)->dqio_sem);return 0 ; out_err:if (oinfo) {
@@ -786,7 +786,7 @@ static int ocfs2_local_read_info(struct super_block *sb, int type)kfree (oinfo); }brelse (bh); -mutex_lock (&sb_dqopt (sb)->dqio_mutex); +down_write (&sb_dqopt (sb)->dqio_sem);return -1 ; }
fs/quota/dquot.c +14 -14
@@ -120,7 +120,7 @@* spinlock to internal buffers before writing. * * Lock ordering (including related VFS locks) is the following: -* s_umount > i_mutex > journal_lock > dquot->dq_lock > dqio_mutex +* s_umount > i_mutex > journal_lock > dquot->dq_lock > dqio_sem */ static __cacheline_aligned_in_smp DEFINE_SPINLOCK (dq_list_lock );
@@ -406,7 +406,7 @@ int dquot_acquire(struct dquot *dquot)struct quota_info *dqopt =sb_dqopt (dquot->dq_sb);mutex_lock (&dquot->dq_lock); -mutex_lock (&dqopt->dqio_mutex); +down_write (&dqopt->dqio_sem);if (!test_bit (DQ_READ_B , &dquot->dq_flags)) ret = dqopt->ops[dquot->dq_id.type]->read_dqblk (dquot);if (ret <0 )
@@ -436,7 +436,7 @@ int dquot_acquire(struct dquot *dquot)smp_mb__before_atomic ();set_bit (DQ_ACTIVE_B , &dquot->dq_flags); out_iolock: -mutex_unlock (&dqopt->dqio_mutex); +up_write (&dqopt->dqio_sem);mutex_unlock (&dquot->dq_lock);return ret; }
@@ -450,7 +450,7 @@ int dquot_commit(struct dquot *dquot)int ret =0 ;struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); -mutex_lock (&dqopt->dqio_mutex); +down_write (&dqopt->dqio_sem);spin_lock (&dq_list_lock);if (!clear_dquot_dirty (dquot)) {spin_unlock (&dq_list_lock);
@@ -464,7 +464,7 @@ int dquot_commit(struct dquot *dquot)else ret = -EIO ; out_sem: -mutex_unlock (&dqopt->dqio_mutex); +up_write (&dqopt->dqio_sem);return ret; }EXPORT_SYMBOL (dquot_commit);
@@ -481,7 +481,7 @@ int dquot_release(struct dquot *dquot)/* Check whether we are not racing with some other dqget() */ if (atomic_read (&dquot->dq_count) >1 ) goto out_dqlock; -mutex_lock (&dqopt->dqio_mutex); +down_write (&dqopt->dqio_sem);if (dqopt->ops[dquot->dq_id.type]->release_dqblk) { ret = dqopt->ops[dquot->dq_id.type]->release_dqblk (dquot);/* Write the info */
@@ -493,7 +493,7 @@ int dquot_release(struct dquot *dquot) ret = ret2; }clear_bit (DQ_ACTIVE_B , &dquot->dq_flags); -mutex_unlock (&dqopt->dqio_mutex); +up_write (&dqopt->dqio_sem); out_dqlock:mutex_unlock (&dquot->dq_lock);return ret;
@@ -2060,9 +2060,9 @@ int dquot_commit_info(struct super_block *sb, int type)int ret;struct quota_info *dqopt =sb_dqopt (sb); -mutex_lock (&dqopt->dqio_mutex); +down_write (&dqopt->dqio_sem); ret = dqopt->ops[type]->write_file_info (sb, type); -mutex_unlock (&dqopt->dqio_mutex); +up_write (&dqopt->dqio_sem);return ret; }EXPORT_SYMBOL (dquot_commit_info);
@@ -2076,9 +2076,9 @@ int dquot_get_next_id(struct super_block *sb, struct kqid *qid)return -ESRCH ;if (!dqopt->ops[qid->type]->get_next_id)return -ENOSYS ; -mutex_lock (&dqopt->dqio_mutex); +down_write (&dqopt->dqio_sem); err = dqopt->ops[qid->type]->get_next_id (sb, qid); -mutex_unlock (&dqopt->dqio_mutex); +up_write (&dqopt->dqio_sem);return err; }EXPORT_SYMBOL (dquot_get_next_id);
@@ -2328,15 +2328,15 @@ static int vfs_load_quota_inode(struct inode *inode, int type, int format_id, dqopt->info[type].dqi_format = fmt; dqopt->info[type].dqi_fmt_id = format_id;INIT_LIST_HEAD (&dqopt->info[type].dqi_dirty_list); -mutex_lock (&dqopt->dqio_mutex); +down_write (&dqopt->dqio_sem); error = dqopt->ops[type]->read_file_info (sb, type);if (error <0 ) { -mutex_unlock (&dqopt->dqio_mutex); +up_write (&dqopt->dqio_sem); goto out_file_init; }if (dqopt->flags &DQUOT_QUOTA_SYS_FILE ) dqopt->info[type].dqi_flags |=DQF_SYS_FILE ; -mutex_unlock (&dqopt->dqio_mutex); +up_write (&dqopt->dqio_sem);spin_lock (&dq_state_lock); dqopt->flags |=dquot_state_flag (flags, type);spin_unlock (&dq_state_lock);
fs/quota/quota_tree.c +1 -1
@@ -379,7 +379,7 @@ int qtree_write_dquot(struct qtree_mem_dqinfo *info, struct dquot *dquot)if (!ddquot)return -ENOMEM ; -/* dq_off is guarded by dqio_mutex */ +/* dq_off is guarded by dqio_sem */ if (!dquot->dq_off) { ret =dq_insert_tree (info, dquot);if (ret <0 ) {
fs/super.c +1 -1
@@ -242,7 +242,7 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags,atomic_set (&s->s_active,1 );mutex_init (&s->s_vfs_rename_mutex);lockdep_set_class (&s->s_vfs_rename_mutex, &type->s_vfs_rename_key); -mutex_init (&s->s_dquot.dqio_mutex); +init_rwsem (&s->s_dquot.dqio_sem); s->s_maxbytes =MAX_NON_LFS ; s->s_op = &default_op; s->s_time_gran =1000000000 ;
include/linux/quota.h +1 -1
@@ -521,7 +521,7 @@ static inline void quota_send_warning(struct kqid qid, dev_t dev,struct quota_info {unsigned int flags;/* Flags for diskquotas on this device */ -struct mutex dqio_mutex;/* lock device while I/O in progress */ +struct rw_semaphore dqio_sem;/* Lock quota file while I/O in progress */ struct inode *files[MAXQUOTAS ];/* inodes of quotafiles */ struct mem_dqinfo info[MAXQUOTAS ];/* Information for each quota type */ const struct quota_format_ops *ops[MAXQUOTAS ];/* Operations for each type */
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 2/27 quota: Do more fine-grained locking in dquot_acquire()
In reply to Jan KaraWe need dqio_sem held just for reading when calling ->read_dqblk() in dquot_acquire(). Also dqio_sem is not needed when setting DQ_READ_B and DQ_ACTIVE_B as concurrent reads and dquot activations are serialized by dq_lock. So acquire and release dqio_sem closer to the place where it is needed. This reduces lock hold time and will make locking changes easier.
--- fs/quota/dquot.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/quota/dquot.c +6 -3
@@ -406,9 +406,11 @@ int dquot_acquire(struct dquot *dquot)struct quota_info *dqopt =sb_dqopt (dquot->dq_sb);mutex_lock (&dquot->dq_lock); -down_write (&dqopt->dqio_sem); -if (!test_bit (DQ_READ_B , &dquot->dq_flags)) +if (!test_bit (DQ_READ_B , &dquot->dq_flags)) { +down_read (&dqopt->dqio_sem); ret = dqopt->ops[dquot->dq_id.type]->read_dqblk (dquot); +up_read (&dqopt->dqio_sem); + }if (ret <0 ) goto out_iolock;/* Make sure flags update is visible after dquot has been filled */
@@ -416,12 +418,14 @@ int dquot_acquire(struct dquot *dquot)set_bit (DQ_READ_B , &dquot->dq_flags);/* Instantiate dquot if needed */ if (!test_bit (DQ_ACTIVE_B , &dquot->dq_flags) && !dquot->dq_off) { +down_write (&dqopt->dqio_sem); ret = dqopt->ops[dquot->dq_id.type]->commit_dqblk (dquot);/* Write the info if needed */ if (info_dirty (&dqopt->info[dquot->dq_id.type])) { ret2 = dqopt->ops[dquot->dq_id.type]->write_file_info ( dquot->dq_sb, dquot->dq_id.type); } +up_write (&dqopt->dqio_sem);if (ret <0 ) goto out_iolock;if (ret2 <0 ) {
@@ -436,7 +440,6 @@ int dquot_acquire(struct dquot *dquot)smp_mb__before_atomic ();set_bit (DQ_ACTIVE_B , &dquot->dq_flags); out_iolock: -up_write (&dqopt->dqio_sem);mutex_unlock (&dquot->dq_lock);return ret; }
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 3/27 quota: Acquire dqio_sem for reading in dquot_get_next_id()
In reply to Jan Karadquot_get_next_id() needs dqio_sem only for reading to protect against racing with modification of quota file structure.
--- fs/quota/dquot.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/quota/dquot.c +2 -2
@@ -2079,9 +2079,9 @@ int dquot_get_next_id(struct super_block *sb, struct kqid *qid)return -ESRCH ;if (!dqopt->ops[qid->type]->get_next_id)return -ENOSYS ; -down_write (&dqopt->dqio_sem); +down_read (&dqopt->dqio_sem); err = dqopt->ops[qid->type]->get_next_id (sb, qid); -up_write (&dqopt->dqio_sem); +up_read (&dqopt->dqio_sem);return err; }EXPORT_SYMBOL (dquot_get_next_id);
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 4/27 quota: Acquire dqio_sem for reading in vfs_load_quota_inode()
In reply to Jan Karavfs_load_quota_inode() needs dqio_sem only for reading. In fact dqio_sem is not needed there at all since the function can be called only during quota on when quota file cannot be modified but let's leave the protection there since it is logical and the path is in no way performance critical.
--- fs/ocfs2/quota_local.c | 6 +++--- fs/quota/dquot.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-)
fs/ocfs2/quota_local.c +3 -3
@@ -693,7 +693,7 @@ static int ocfs2_local_read_info(struct super_block *sb, int type)/* We don't need the lock and we have to acquire quota file locks * which will later depend on this lock */ -up_write (&sb_dqopt (sb)->dqio_sem); +up_read (&sb_dqopt (sb)->dqio_sem); info->dqi_max_spc_limit =0x7fffffffffffffffLL ; info->dqi_max_ino_limit =0x7fffffffffffffffLL ; oinfo =kmalloc (sizeof (struct ocfs2_mem_dqinfo ),GFP_NOFS );
@@ -772,7 +772,7 @@ static int ocfs2_local_read_info(struct super_block *sb, int type) goto out_err; } -down_write (&sb_dqopt (sb)->dqio_sem); +down_read (&sb_dqopt (sb)->dqio_sem);return 0 ; out_err:if (oinfo) {
@@ -786,7 +786,7 @@ static int ocfs2_local_read_info(struct super_block *sb, int type)kfree (oinfo); }brelse (bh); -down_write (&sb_dqopt (sb)->dqio_sem); +down_read (&sb_dqopt (sb)->dqio_sem);return -1 ; }
fs/quota/dquot.c +3 -3
@@ -2331,15 +2331,15 @@ static int vfs_load_quota_inode(struct inode *inode, int type, int format_id, dqopt->info[type].dqi_format = fmt; dqopt->info[type].dqi_fmt_id = format_id;INIT_LIST_HEAD (&dqopt->info[type].dqi_dirty_list); -down_write (&dqopt->dqio_sem); +down_read (&dqopt->dqio_sem); error = dqopt->ops[type]->read_file_info (sb, type);if (error <0 ) { -up_write (&dqopt->dqio_sem); +up_read (&dqopt->dqio_sem); goto out_file_init; }if (dqopt->flags &DQUOT_QUOTA_SYS_FILE ) dqopt->info[type].dqi_flags |=DQF_SYS_FILE ; -up_write (&dqopt->dqio_sem); +up_read (&dqopt->dqio_sem);spin_lock (&dq_state_lock); dqopt->flags |=dquot_state_flag (flags, type);spin_unlock (&dq_state_lock);
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 5/27 quota: Protect dquot writeout with dq_lock
In reply to Jan KaraCurrently dquot writeout is only protected by dqio_sem held for writing. As we transition to a finer grained locking we will use dquot->dq_lock instead. So acquire it in dquot_commit() and move dqio_sem just around ->commit_dqblk() call as it is still needed to serialize quota file changes.
--- fs/quota/dquot.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-)
fs/quota/dquot.c +14 -14
@@ -110,14 +110,11 @@* sure they cannot race with quotaon which first sets S_NOQUOTA flag and * then drops all pointers to dquots from an inode. * -* Each dquot has its dq_lock mutex. Locked dquots might not be referenced -* from inodes (dquot_alloc_space() and such don't check the dq_lock). -* Currently dquot is locked only when it is being read to memory (or space for -* it is being allocated) on the first dqget() and when it is being released on -* the last dqput(). The allocation and release oparations are serialized by -* the dq_lock and by checking the use count in dquot_release(). Write -* operations on dquots don't hold dq_lock as they copy data under dq_data_lock -* spinlock to internal buffers before writing. +* Each dquot has its dq_lock mutex. Dquot is locked when it is being read to +* memory (or space for it is being allocated) on the first dqget(), when it is +* being written out, and when it is being released on the last dqput(). The +* allocation and release operations are serialized by the dq_lock and by +* checking the use count in dquot_release(). * * Lock ordering (including related VFS locks) is the following: * s_umount > i_mutex > journal_lock > dquot->dq_lock > dqio_sem
@@ -453,21 +450,24 @@ int dquot_commit(struct dquot *dquot)int ret =0 ;struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); -down_write (&dqopt->dqio_sem); +mutex_lock (&dquot->dq_lock);spin_lock (&dq_list_lock);if (!clear_dquot_dirty (dquot)) {spin_unlock (&dq_list_lock); - goto out_sem; + goto out_lock; }spin_unlock (&dq_list_lock);/* Inactive dquot can be only if there was error during read/init * => we have better not writing it */ -if (test_bit (DQ_ACTIVE_B , &dquot->dq_flags)) +if (test_bit (DQ_ACTIVE_B , &dquot->dq_flags)) { +down_write (&dqopt->dqio_sem); ret = dqopt->ops[dquot->dq_id.type]->commit_dqblk (dquot); -else +up_write (&dqopt->dqio_sem); + }else { ret = -EIO ; -out_sem: -up_write (&dqopt->dqio_sem); + } +out_lock: +mutex_unlock (&dquot->dq_lock);return ret; }EXPORT_SYMBOL (dquot_commit);
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 6/27 quota: Push dqio_sem down to ->read_dqblk()
In reply to Jan KaraPush down acquisition of dqio_sem into ->read_dqblk() callback. It will allow quota formats to decide whether they need it or not.
--- fs/quota/dquot.c | 5 +---- fs/quota/quota_v1.c | 5 ++++- fs/quota/quota_v2.c | 10 +++++++++- 3 files changed, 14 insertions(+), 6 deletions(-)
fs/quota/dquot.c +1 -4
@@ -403,11 +403,8 @@ int dquot_acquire(struct dquot *dquot)struct quota_info *dqopt =sb_dqopt (dquot->dq_sb);mutex_lock (&dquot->dq_lock); -if (!test_bit (DQ_READ_B , &dquot->dq_flags)) { -down_read (&dqopt->dqio_sem); +if (!test_bit (DQ_READ_B , &dquot->dq_flags)) ret = dqopt->ops[dquot->dq_id.type]->read_dqblk (dquot); -up_read (&dqopt->dqio_sem); - }if (ret <0 ) goto out_iolock;/* Make sure flags update is visible after dquot has been filled */
fs/quota/quota_v1.c +4 -1
@@ -56,10 +56,12 @@ static int v1_read_dqblk(struct dquot *dquot) {int type = dquot->dq_id.type;struct v1_disk_dqblk dqblk; +struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); -if (!sb_dqopt (dquot->dq_sb)->files[type]) +if (!dqopt->files[type])return -EINVAL ; +down_read (&dqopt->dqio_sem);/* Set structure to 0s in case read fails/is after end of file */ memset (&dqblk,0 ,sizeof (struct v1_disk_dqblk )); dquot->dq_sb->s_op->quota_read(dquot->dq_sb, type, (char *)&dqblk,
@@ -73,6 +75,7 @@ static int v1_read_dqblk(struct dquot *dquot) dquot->dq_dqb.dqb_isoftlimit == 0)set_bit (DQ_FAKE_B , &dquot->dq_flags);dqstats_inc (DQST_READS ); +up_read (&dqopt->dqio_sem);return 0 ; }
fs/quota/quota_v2.c +9 -1
@@ -285,7 +285,15 @@ static int v2r1_is_id(void *dp, struct dquot *dquot)static int v2_read_dquot (struct dquot *dquot) { -return qtree_read_dquot (sb_dqinfo (dquot->dq_sb, dquot->dq_id.type)->dqi_priv, dquot); +struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); +int ret; + +down_read (&dqopt->dqio_sem); + ret =qtree_read_dquot ( +sb_dqinfo (dquot->dq_sb, dquot->dq_id.type)->dqi_priv, + dquot); +up_read (&dqopt->dqio_sem); +return ret; }static int v2_write_dquot (struct dquot *dquot)
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 7/27 quota: Remove locking for reading from the old quota format
In reply to Jan KaraThe old quota format has fixed offset in quota file based on ID so there's no locking needed against concurrent modifications of the file (locking against concurrent IO on the same dquot is still provided by dq_lock).
--- fs/quota/quota_v1.c | 2 -- 1 file changed, 2 deletions(-)
fs/quota/quota_v1.c +0 -2
@@ -61,7 +61,6 @@ static int v1_read_dqblk(struct dquot *dquot)if (!dqopt->files[type])return -EINVAL ; -down_read (&dqopt->dqio_sem);/* Set structure to 0s in case read fails/is after end of file */ memset (&dqblk,0 ,sizeof (struct v1_disk_dqblk )); dquot->dq_sb->s_op->quota_read(dquot->dq_sb, type, (char *)&dqblk,
@@ -75,7 +74,6 @@ static int v1_read_dqblk(struct dquot *dquot) dquot->dq_dqb.dqb_isoftlimit == 0)set_bit (DQ_FAKE_B , &dquot->dq_flags);dqstats_inc (DQST_READS ); -up_read (&dqopt->dqio_sem);return 0 ; }
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 8/27 quota: Push dqio_sem down to ->write_dqblk()
In reply to Jan KaraPush down acquisition of dqio_sem into ->write_dqblk() callback. It will allow quota formats to decide whether they need it or not.
--- fs/quota/dquot.c | 11 ++++------- fs/quota/quota_v1.c | 3 +++ fs/quota/quota_v2.c | 10 +++++++++- 3 files changed, 16 insertions(+), 8 deletions(-)
fs/quota/dquot.c +4 -7
@@ -412,14 +412,14 @@ int dquot_acquire(struct dquot *dquot)set_bit (DQ_READ_B , &dquot->dq_flags);/* Instantiate dquot if needed */ if (!test_bit (DQ_ACTIVE_B , &dquot->dq_flags) && !dquot->dq_off) { -down_write (&dqopt->dqio_sem); ret = dqopt->ops[dquot->dq_id.type]->commit_dqblk (dquot);/* Write the info if needed */ if (info_dirty (&dqopt->info[dquot->dq_id.type])) { +down_write (&dqopt->dqio_sem); ret2 = dqopt->ops[dquot->dq_id.type]->write_file_info ( dquot->dq_sb, dquot->dq_id.type); +up_write (&dqopt->dqio_sem); } -up_write (&dqopt->dqio_sem);if (ret <0 ) goto out_iolock;if (ret2 <0 ) {
@@ -456,13 +456,10 @@ int dquot_commit(struct dquot *dquot)spin_unlock (&dq_list_lock);/* Inactive dquot can be only if there was error during read/init * => we have better not writing it */ -if (test_bit (DQ_ACTIVE_B , &dquot->dq_flags)) { -down_write (&dqopt->dqio_sem); +if (test_bit (DQ_ACTIVE_B , &dquot->dq_flags)) ret = dqopt->ops[dquot->dq_id.type]->commit_dqblk (dquot); -up_write (&dqopt->dqio_sem); - }else { +else ret = -EIO ; - } out_lock:mutex_unlock (&dquot->dq_lock);return ret;
fs/quota/quota_v1.c +3 -0
@@ -83,7 +83,9 @@ static int v1_commit_dqblk(struct dquot *dquot)short type = dquot->dq_id.type;ssize_t ret;struct v1_disk_dqblk dqblk; +struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); +down_write (&dqopt->dqio_sem);v1_mem2disk_dqblk (&dqblk, &dquot->dq_dqb);if (((type ==USRQUOTA ) &&uid_eq (dquot->dq_id.uid,GLOBAL_ROOT_UID )) || ((type ==GRPQUOTA ) &&gid_eq (dquot->dq_id.gid,GLOBAL_ROOT_GID ))) {
@@ -97,6 +99,7 @@ static int v1_commit_dqblk(struct dquot *dquot) ret = dquot->dq_sb->s_op->quota_write (dquot->dq_sb, type, (char *)&dqblk,sizeof (struct v1_disk_dqblk ),v1_dqoff (from_kqid (&init_user_ns, dquot->dq_id))); +up_write (&dqopt->dqio_sem);if (ret !=sizeof (struct v1_disk_dqblk )) {quota_error (dquot->dq_sb,"dquota write failed" );if (ret >=0 )
fs/quota/quota_v2.c +9 -1
@@ -298,7 +298,15 @@ static int v2_read_dquot(struct dquot *dquot)static int v2_write_dquot (struct dquot *dquot) { -return qtree_write_dquot (sb_dqinfo (dquot->dq_sb, dquot->dq_id.type)->dqi_priv, dquot); +struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); +int ret; + +down_write (&dqopt->dqio_sem); + ret =qtree_write_dquot ( +sb_dqinfo (dquot->dq_sb, dquot->dq_id.type)->dqi_priv, + dquot); +up_write (&dqopt->dqio_sem); +return ret; }static int v2_release_dquot (struct dquot *dquot)
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 9/27 quota: Do not acquire dqio_sem for dquot overwrites in v2 format
In reply to Jan KaraWhen dquot has space already allocated in a quota file, we just overwrite that place when writing dquot. So we don't need any protection against other modifications of quota file as these keep dquot in place.
--- fs/quota/quota_v2.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
fs/quota/quota_v2.c +14 -3
@@ -300,12 +300,23 @@ static int v2_write_dquot(struct dquot *dquot) {struct quota_info *dqopt =sb_dqopt (dquot->dq_sb);int ret; - -down_write (&dqopt->dqio_sem); +bool alloc = false; + +/* +* If space for dquot is already allocated, we don't need any +* protection as we'll only overwrite the place of dquot. We are +* still protected by concurrent writes of the same dquot by +* dquot->dq_lock. +*/ +if (!dquot->dq_off) { + alloc = true; +down_write (&dqopt->dqio_sem); + } ret =qtree_write_dquot (sb_dqinfo (dquot->dq_sb, dquot->dq_id.type)->dqi_priv, dquot); -up_write (&dqopt->dqio_sem); +if (alloc) +up_write (&dqopt->dqio_sem);return ret; }
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 10/27 quota: Remove locking for writing to the old quota format
In reply to Jan KaraThe old quota quota format has fixed offset in quota file based on ID so there's no locking needed against concurrent modifications of the file (locking against concurrent IO on the same dquot is still provided by dq_lock).
--- fs/quota/quota_v1.c | 3 --- 1 file changed, 3 deletions(-)
fs/quota/quota_v1.c +0 -3
@@ -83,9 +83,7 @@ static int v1_commit_dqblk(struct dquot *dquot)short type = dquot->dq_id.type;ssize_t ret;struct v1_disk_dqblk dqblk; -struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); -down_write (&dqopt->dqio_sem);v1_mem2disk_dqblk (&dqblk, &dquot->dq_dqb);if (((type ==USRQUOTA ) &&uid_eq (dquot->dq_id.uid,GLOBAL_ROOT_UID )) || ((type ==GRPQUOTA ) &&gid_eq (dquot->dq_id.gid,GLOBAL_ROOT_GID ))) {
@@ -99,7 +97,6 @@ static int v1_commit_dqblk(struct dquot *dquot) ret = dquot->dq_sb->s_op->quota_write (dquot->dq_sb, type, (char *)&dqblk,sizeof (struct v1_disk_dqblk ),v1_dqoff (from_kqid (&init_user_ns, dquot->dq_id))); -up_write (&dqopt->dqio_sem);if (ret !=sizeof (struct v1_disk_dqblk )) {quota_error (dquot->dq_sb,"dquota write failed" );if (ret >=0 )
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 11/27 quota: Push dqio_sem down to ->release_dqblk()
In reply to Jan KaraPush down acquisition of dqio_sem into ->release_dqblk() callback. It will allow quota formats to decide whether they need it or not.
--- fs/quota/dquot.c | 4 ++-- fs/quota/quota_v2.c | 9 ++++++++- 2 files changed, 10 insertions(+), 3 deletions(-)
fs/quota/dquot.c +2 -2
@@ -478,19 +478,19 @@ int dquot_release(struct dquot *dquot)/* Check whether we are not racing with some other dqget() */ if (atomic_read (&dquot->dq_count) >1 ) goto out_dqlock; -down_write (&dqopt->dqio_sem);if (dqopt->ops[dquot->dq_id.type]->release_dqblk) { ret = dqopt->ops[dquot->dq_id.type]->release_dqblk (dquot);/* Write the info */ if (info_dirty (&dqopt->info[dquot->dq_id.type])) { +down_write (&dqopt->dqio_sem); ret2 = dqopt->ops[dquot->dq_id.type]->write_file_info ( dquot->dq_sb, dquot->dq_id.type); +up_write (&dqopt->dqio_sem); }if (ret >=0 ) ret = ret2; }clear_bit (DQ_ACTIVE_B , &dquot->dq_flags); -up_write (&dqopt->dqio_sem); out_dqlock:mutex_unlock (&dquot->dq_lock);return ret;
fs/quota/quota_v2.c +8 -1
@@ -322,7 +322,14 @@ static int v2_write_dquot(struct dquot *dquot)static int v2_release_dquot (struct dquot *dquot) { -return qtree_release_dquot (sb_dqinfo (dquot->dq_sb, dquot->dq_id.type)->dqi_priv, dquot); +struct quota_info *dqopt =sb_dqopt (dquot->dq_sb); +int ret; + +down_write (&dqopt->dqio_sem); + ret =qtree_release_dquot (sb_dqinfo (dquot->dq_sb, dquot->dq_id.type)->dqi_priv, dquot); +up_write (&dqopt->dqio_sem); + +return ret; }static int v2_free_file_info (struct super_block *sb,int type)
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 12/27 quota: Push dqio_sem down to ->get_next_id()
In reply to Jan KaraPush down acquisition of dqio_sem into ->get_next_id() callback. Mostly for consistency with other operations.
--- fs/quota/dquot.c | 6 +----- fs/quota/quota_v2.c | 8 +++++++- 2 files changed, 8 insertions(+), 6 deletions(-)
fs/quota/dquot.c +1 -5
@@ -2067,16 +2067,12 @@ EXPORT_SYMBOL(dquot_commit_info);int dquot_get_next_id (struct super_block *sb,struct kqid *qid) {struct quota_info *dqopt =sb_dqopt (sb); -int err;if (!sb_has_quota_active (sb, qid->type))return -ESRCH ;if (!dqopt->ops[qid->type]->get_next_id)return -ENOSYS ; -down_read (&dqopt->dqio_sem); - err = dqopt->ops[qid->type]->get_next_id (sb, qid); -up_read (&dqopt->dqio_sem); -return err; +return dqopt->ops[qid->type]->get_next_id (sb, qid); }EXPORT_SYMBOL (dquot_get_next_id);
fs/quota/quota_v2.c +7 -1
@@ -340,7 +340,13 @@ static int v2_free_file_info(struct super_block *sb, int type)static int v2_get_next_id (struct super_block *sb,struct kqid *qid) { -return qtree_get_next_id (sb_dqinfo (sb, qid->type)->dqi_priv, qid); +struct quota_info *dqopt =sb_dqopt (sb); +int ret; + +down_read (&dqopt->dqio_sem); + ret =qtree_get_next_id (sb_dqinfo (sb, qid->type)->dqi_priv, qid); +up_read (&dqopt->dqio_sem); +return ret; }static const struct quota_format_ops v2_format_ops = {
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 13/27 quota: Push dqio_sem down to ->write_file_info()
In reply to Jan KaraPush down acquisition of dqio_sem into ->write_file_info() callback. Mostly for consistency with other operations.
--- fs/ocfs2/quota_global.c | 8 ++++++-- fs/quota/dquot.c | 10 +--------- fs/quota/quota_v1.c | 2 ++ fs/quota/quota_v2.c | 5 ++++- 4 files changed, 13 insertions(+), 12 deletions(-)
fs/ocfs2/quota_global.c +6 -2
@@ -443,13 +443,17 @@ static int __ocfs2_global_write_info(struct super_block *sb, int type)int ocfs2_global_write_info (struct super_block *sb,int type) {int err; -struct ocfs2_mem_dqinfo *info =sb_dqinfo (sb, type)->dqi_priv; +struct quota_info *dqopt =sb_dqopt (sb); +struct ocfs2_mem_dqinfo *info = dqopt->info[type].dqi_priv; +down_write (&dqopt->dqio_sem); err =ocfs2_qinfo_lock (info,1 );if (err <0 ) -return err; + goto out_sem; err =__ocfs2_global_write_info (sb, type);ocfs2_qinfo_unlock (info,1 ); +out_sem: +up_write (&dqopt->dqio_sem);return err; }
fs/quota/dquot.c +1 -9
@@ -415,10 +415,8 @@ int dquot_acquire(struct dquot *dquot) ret = dqopt->ops[dquot->dq_id.type]->commit_dqblk (dquot);/* Write the info if needed */ if (info_dirty (&dqopt->info[dquot->dq_id.type])) { -down_write (&dqopt->dqio_sem); ret2 = dqopt->ops[dquot->dq_id.type]->write_file_info ( dquot->dq_sb, dquot->dq_id.type); -up_write (&dqopt->dqio_sem); }if (ret <0 ) goto out_iolock;
@@ -482,10 +480,8 @@ int dquot_release(struct dquot *dquot) ret = dqopt->ops[dquot->dq_id.type]->release_dqblk (dquot);/* Write the info */ if (info_dirty (&dqopt->info[dquot->dq_id.type])) { -down_write (&dqopt->dqio_sem); ret2 = dqopt->ops[dquot->dq_id.type]->write_file_info ( dquot->dq_sb, dquot->dq_id.type); -up_write (&dqopt->dqio_sem); }if (ret >=0 ) ret = ret2;
@@ -2054,13 +2050,9 @@ EXPORT_SYMBOL(dquot_transfer);*/ int dquot_commit_info (struct super_block *sb,int type) { -int ret;struct quota_info *dqopt =sb_dqopt (sb); -down_write (&dqopt->dqio_sem); - ret = dqopt->ops[type]->write_file_info (sb, type); -up_write (&dqopt->dqio_sem); -return ret; +return dqopt->ops[type]->write_file_info (sb, type); }EXPORT_SYMBOL (dquot_commit_info);
fs/quota/quota_v1.c +2 -0
@@ -186,6 +186,7 @@ static int v1_write_file_info(struct super_block *sb, int type)struct v1_disk_dqblk dqblk;int ret; +down_write (&dqopt->dqio_sem); dqopt->info[type].dqi_flags &= ~DQF_INFO_DIRTY ; ret = sb->s_op->quota_read (sb, type, (char *)&dqblk,sizeof (struct v1_disk_dqblk ),v1_dqoff (0 ));
@@ -203,6 +204,7 @@ static int v1_write_file_info(struct super_block *sb, int type)else if (ret >0 ) ret = -EIO ; out: +up_write (&dqopt->dqio_sem);return ret; }
fs/quota/quota_v2.c +4 -1
@@ -154,10 +154,12 @@ static int v2_read_file_info(struct super_block *sb, int type)static int v2_write_file_info (struct super_block *sb,int type) {struct v2_disk_dqinfo dinfo; -struct mem_dqinfo *info =sb_dqinfo (sb, type); +struct quota_info *dqopt =sb_dqopt (sb); +struct mem_dqinfo *info = &dqopt->info[type];struct qtree_mem_dqinfo *qinfo = info->dqi_priv;ssize_t size; +down_write (&dqopt->dqio_sem);spin_lock (&dq_data_lock); info->dqi_flags &= ~DQF_INFO_DIRTY ; dinfo.dqi_bgrace =cpu_to_le32 (info->dqi_bgrace);
@@ -170,6 +172,7 @@ static int v2_write_file_info(struct super_block *sb, int type) dinfo.dqi_free_entry =cpu_to_le32 (qinfo->dqi_free_entry); size = sb->s_op->quota_write (sb, type, (char *)&dinfo,sizeof (struct v2_disk_dqinfo ),V2_DQINFOOFF ); +up_write (&dqopt->dqio_sem);if (size !=sizeof (struct v2_disk_dqinfo )) {quota_error (sb,"Can't write info structure" );return -1 ;
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 14/27 quota: Push dqio_sem down to ->read_file_info()
In reply to Jan KaraPush down acquisition of dqio_sem into ->read_file_info() callback. This is for consistency with other operations and it also allows us to get rid of an ugliness in OCFS2.
--- fs/ocfs2/quota_local.c | 5 ----- fs/quota/dquot.c | 6 +----- fs/quota/quota_v1.c | 2 ++ fs/quota/quota_v2.c | 28 ++++++++++++++++++++-------- 4 files changed, 23 insertions(+), 18 deletions(-)
fs/ocfs2/quota_local.c +0 -5
@@ -691,9 +691,6 @@ static int ocfs2_local_read_info(struct super_block *sb, int type)struct ocfs2_quota_recovery *rec;int locked =0 ; -/* We don't need the lock and we have to acquire quota file locks -* which will later depend on this lock */ -up_read (&sb_dqopt (sb)->dqio_sem); info->dqi_max_spc_limit =0x7fffffffffffffffLL ; info->dqi_max_ino_limit =0x7fffffffffffffffLL ; oinfo =kmalloc (sizeof (struct ocfs2_mem_dqinfo ),GFP_NOFS );
@@ -772,7 +769,6 @@ static int ocfs2_local_read_info(struct super_block *sb, int type) goto out_err; } -down_read (&sb_dqopt (sb)->dqio_sem);return 0 ; out_err:if (oinfo) {
@@ -786,7 +782,6 @@ static int ocfs2_local_read_info(struct super_block *sb, int type)kfree (oinfo); }brelse (bh); -down_read (&sb_dqopt (sb)->dqio_sem);return -1 ; }
fs/quota/dquot.c +1 -5
@@ -2313,15 +2313,11 @@ static int vfs_load_quota_inode(struct inode *inode, int type, int format_id, dqopt->info[type].dqi_format = fmt; dqopt->info[type].dqi_fmt_id = format_id;INIT_LIST_HEAD (&dqopt->info[type].dqi_dirty_list); -down_read (&dqopt->dqio_sem); error = dqopt->ops[type]->read_file_info (sb, type); -if (error <0 ) { -up_read (&dqopt->dqio_sem); +if (error <0 ) goto out_file_init; - }if (dqopt->flags &DQUOT_QUOTA_SYS_FILE ) dqopt->info[type].dqi_flags |=DQF_SYS_FILE ; -up_read (&dqopt->dqio_sem);spin_lock (&dq_state_lock); dqopt->flags |=dquot_state_flag (flags, type);spin_unlock (&dq_state_lock);
fs/quota/quota_v1.c +2 -0
@@ -161,6 +161,7 @@ static int v1_read_file_info(struct super_block *sb, int type)struct v1_disk_dqblk dqblk;int ret; +down_read (&dqopt->dqio_sem); ret = sb->s_op->quota_read (sb, type, (char *)&dqblk,sizeof (struct v1_disk_dqblk ),v1_dqoff (0 ));if (ret !=sizeof (struct v1_disk_dqblk )) {
@@ -177,6 +178,7 @@ static int v1_read_file_info(struct super_block *sb, int type) dqopt->info[type].dqi_bgrace = dqblk.dqb_btime ? dqblk.dqb_btime :MAX_DQ_TIME ; out: +up_read (&dqopt->dqio_sem);return ret; }
fs/quota/quota_v2.c +20 -8
@@ -90,29 +90,38 @@ static int v2_read_file_info(struct super_block *sb, int type) {struct v2_disk_dqinfo dinfo;struct v2_disk_dqheader dqhead; -struct mem_dqinfo *info =sb_dqinfo (sb, type); +struct quota_info *dqopt =sb_dqopt (sb); +struct mem_dqinfo *info = &dqopt->info[type];struct qtree_mem_dqinfo *qinfo;ssize_t size;unsigned int version; +int ret; -if (!v2_read_header (sb, type, &dqhead)) -return -1 ; +down_read (&dqopt->dqio_sem); +if (!v2_read_header (sb, type, &dqhead)) { + ret =-1 ; + goto out; + } version =le32_to_cpu (dqhead.dqh_version);if ((info->dqi_fmt_id ==QFMT_VFS_V0 && version !=0 ) || - (info->dqi_fmt_id ==QFMT_VFS_V1 && version !=1 )) -return -1 ; + (info->dqi_fmt_id ==QFMT_VFS_V1 && version !=1 )) { + ret =-1 ; + goto out; + } size = sb->s_op->quota_read (sb, type, (char *)&dinfo,sizeof (struct v2_disk_dqinfo ),V2_DQINFOOFF );if (size !=sizeof (struct v2_disk_dqinfo )) {quota_error (sb,"Can't read info structure" ); -return -1 ; + ret =-1 ; + goto out; } info->dqi_priv =kmalloc (sizeof (struct qtree_mem_dqinfo ),GFP_NOFS );if (!info->dqi_priv) {printk (KERN_WARNING "Not enough memory for quota information structure.\n" ); -return -ENOMEM ; + ret = -ENOMEM ; + goto out; } qinfo = info->dqi_priv;if (version ==0 ) {
@@ -147,7 +156,10 @@ static int v2_read_file_info(struct super_block *sb, int type) qinfo->dqi_entry_size =sizeof (struct v2r1_disk_dqblk ); qinfo->dqi_ops = &v2r1_qtree_ops; } -return 0 ; + ret =0 ; +out: +up_read (&dqopt->dqio_sem); +return ret; }/* Write information header to quota file */
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 15/27 quota: Fix error codes in v2_read_file_info()
In reply to Jan Karav2_read_file_info() returned -1 instead of proper error codes on error. Luckily this is not easily visible from userspace as we have called ->check_quota_file shortly before and thus already verified the quota file is sane. Still set the error codes to proper values.
--- fs/quota/quota_v2.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/quota/quota_v2.c +3 -3
@@ -99,13 +99,13 @@ static int v2_read_file_info(struct super_block *sb, int type)down_read (&dqopt->dqio_sem);if (!v2_read_header (sb, type, &dqhead)) { - ret =-1 ; + ret = -EIO ; goto out; } version =le32_to_cpu (dqhead.dqh_version);if ((info->dqi_fmt_id ==QFMT_VFS_V0 && version !=0 ) || (info->dqi_fmt_id ==QFMT_VFS_V1 && version !=1 )) { - ret =-1 ; + ret = -EINVAL ; goto out; }
@@ -113,7 +113,7 @@ static int v2_read_file_info(struct super_block *sb, int type) sizeof(struct v2_disk_dqinfo), V2_DQINFOOFF);if (size !=sizeof (struct v2_disk_dqinfo )) {quota_error (sb,"Can't read info structure" ); - ret =-1 ; + ret = -EIO ; goto out; } info->dqi_priv =kmalloc (sizeof (struct qtree_mem_dqinfo ),GFP_NOFS );
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 16/27 quota: Propagate ->quota_read errors from v2_read_file_info()
In reply to Jan KaraCurrently we return -EIO on any error (or short read) from ->quota_read() while reading quota info. Propagate the error code instead.
--- fs/quota/quota_v2.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
fs/quota/quota_v2.c +11 -7
@@ -65,9 +65,11 @@ static int v2_read_header(struct super_block *sb, int type,if (size !=sizeof (struct v2_disk_dqheader )) {quota_error (sb,"Failed header read: expected=%zd got=%zd" ,sizeof (struct v2_disk_dqheader ), size); -return 0 ; +if (size <0 ) +return size; +return -EIO ; } -return 1 ; +return 0 ; }/* Check whether given file is really vfsv0 quotafile */
@@ -77,7 +79,7 @@ static int v2_check_quota_file(struct super_block *sb, int type)static const uint quota_magics[] =V2_INITQMAGICS ;static const uint quota_versions[] =V2_INITQVERSIONS ; -if (!v2_read_header (sb, type, &dqhead)) +if (v2_read_header (sb, type, &dqhead))return 0 ;if (le32_to_cpu (dqhead.dqh_magic) != quota_magics[type] ||le32_to_cpu (dqhead.dqh_version) > quota_versions[type])
@@ -98,10 +100,9 @@ static int v2_read_file_info(struct super_block *sb, int type)int ret;down_read (&dqopt->dqio_sem); -if (!v2_read_header (sb, type, &dqhead)) { - ret = -EIO ; + ret =v2_read_header (sb, type, &dqhead); +if (ret <0 ) goto out; - } version =le32_to_cpu (dqhead.dqh_version);if ((info->dqi_fmt_id ==QFMT_VFS_V0 && version !=0 ) || (info->dqi_fmt_id ==QFMT_VFS_V1 && version !=1 )) {
@@ -113,7 +114,10 @@ static int v2_read_file_info(struct super_block *sb, int type) sizeof(struct v2_disk_dqinfo), V2_DQINFOOFF);if (size !=sizeof (struct v2_disk_dqinfo )) {quota_error (sb,"Can't read info structure" ); - ret = -EIO ; +if (size <0 ) + ret = size; +else + ret = -EIO ; goto out; } info->dqi_priv =kmalloc (sizeof (struct qtree_mem_dqinfo ),GFP_NOFS );
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 17/27 quota: Fix possible corruption of dqi_flags
In reply to Jan Karadqi_flags modifications are protected by dq_data_lock. However the modifications in vfs_load_quota_inode() and in mark_info_dirty() were not which could lead to corruption of dqi_flags. Since modifications to dqi_flags are rare, this is hard to observe in practice but in theory it could happen. Fix the problem by always using dq_data_lock for protection.
--- fs/quota/dquot.c | 9 +++++++-- fs/quota/quota_v1.c | 4 +++- 2 files changed, 10 insertions(+), 3 deletions(-)
fs/quota/dquot.c +7 -2
@@ -389,7 +389,9 @@ static inline int clear_dquot_dirty(struct dquot *dquot)void mark_info_dirty (struct super_block *sb,int type) { -set_bit (DQF_INFO_DIRTY_B , &sb_dqopt (sb)->info[type].dqi_flags); +spin_lock (&dq_data_lock); +sb_dqopt (sb)->info[type].dqi_flags |=DQF_INFO_DIRTY ; +spin_unlock (&dq_data_lock); }EXPORT_SYMBOL (mark_info_dirty);
@@ -2316,8 +2318,11 @@ static int vfs_load_quota_inode(struct inode *inode, int type, int format_id, error = dqopt->ops[type]->read_file_info (sb, type);if (error <0 ) goto out_file_init; -if (dqopt->flags &DQUOT_QUOTA_SYS_FILE ) +if (dqopt->flags &DQUOT_QUOTA_SYS_FILE ) { +spin_lock (&dq_data_lock); dqopt->info[type].dqi_flags |=DQF_SYS_FILE ; +spin_unlock (&dq_data_lock); + }spin_lock (&dq_state_lock); dqopt->flags |=dquot_state_flag (flags, type);spin_unlock (&dq_state_lock);
fs/quota/quota_v1.c +3 -1
@@ -189,7 +189,6 @@ static int v1_write_file_info(struct super_block *sb, int type)int ret;down_write (&dqopt->dqio_sem); - dqopt->info[type].dqi_flags &= ~DQF_INFO_DIRTY ; ret = sb->s_op->quota_read (sb, type, (char *)&dqblk,sizeof (struct v1_disk_dqblk ),v1_dqoff (0 ));if (ret !=sizeof (struct v1_disk_dqblk )) {
@@ -197,8 +196,11 @@ static int v1_write_file_info(struct super_block *sb, int type) ret = -EIO ; goto out; } +spin_lock (&dq_data_lock); + dqopt->info[type].dqi_flags &= ~DQF_INFO_DIRTY ; dqblk.dqb_itime = dqopt->info[type].dqi_igrace; dqblk.dqb_btime = dqopt->info[type].dqi_bgrace; +spin_unlock (&dq_data_lock); ret = sb->s_op->quota_write (sb, type, (char *)&dqblk,sizeof (struct v1_disk_dqblk ),v1_dqoff (0 ));if (ret ==sizeof (struct v1_disk_dqblk ))
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 18/27 quota: Do not dirty bad dquots
In reply to Jan KaraCurrently we mark dirty even dquots that are not active (i.e., initialization or reading failed for them). Thus later we have to check whether dirty dquot is really active and just clear the dirty bit if not. Avoid this complication by just never marking non-active dquot as dirty.
--- fs/quota/dquot.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
fs/quota/dquot.c +7 -8
@@ -339,6 +339,9 @@ int dquot_mark_dquot_dirty(struct dquot *dquot) {int ret =1 ; +if (!test_bit (DQ_ACTIVE_B , &dquot->dq_flags)) +return 0 ; +/* If quota is dirty already, we don't have to acquire dq_list_lock */ if (test_bit (DQ_MOD_B , &dquot->dq_flags))return 1 ;
@@ -624,11 +627,9 @@ int dquot_writeback_dquots(struct super_block *sb, int type)while (!list_empty (dirty)) { dquot = list_first_entry(dirty, struct dquot, dq_dirty); -/* Dirty and inactive can be only bad dquot... */ -if (!test_bit (DQ_ACTIVE_B , &dquot->dq_flags)) { -clear_dquot_dirty (dquot); -continue ; - } + +WARN_ON (!test_bit (DQ_ACTIVE_B , &dquot->dq_flags)); +/* Now we have active dquot from which someone is * holding reference so we can safely just increase * use count */
@@ -759,7 +760,7 @@ void dqput(struct dquot *dquot)return ; }/* Need to release dquot? */ -if (test_bit (DQ_ACTIVE_B , &dquot->dq_flags) &&dquot_dirty (dquot)) { +if (dquot_dirty (dquot)) {spin_unlock (&dq_list_lock);/* Commit dquot before releasing */ ret = dquot->dq_sb->dq_op->write_dquot (dquot);
@@ -777,8 +778,6 @@ void dqput(struct dquot *dquot) } goto we_slept; } -/* Clear flag in case dquot was inactive (something bad happened) */ -clear_dquot_dirty (dquot);if (test_bit (DQ_ACTIVE_B , &dquot->dq_flags)) {spin_unlock (&dq_list_lock); dquot->dq_sb->dq_op->release_dquot (dquot);
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 19/27 quota: Move locking into clear_dquot_dirty()
In reply to Jan KaraMove locking of dq_list_lock into clear_dquot_dirty(). It makes the function more self-contained and will simplify our life later.
--- fs/quota/dquot.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)
fs/quota/dquot.c +6 -9
@@ -381,12 +381,15 @@ static inline void dqput_all(struct dquot **dquot)dqput (dquot[cnt]); } -/* This function needs dq_list_lock */ static inline int clear_dquot_dirty (struct dquot *dquot) { -if (!test_and_clear_bit (DQ_MOD_B , &dquot->dq_flags)) +spin_lock (&dq_list_lock); +if (!test_and_clear_bit (DQ_MOD_B , &dquot->dq_flags)) { +spin_unlock (&dq_list_lock);return 0 ; + }list_del_init (&dquot->dq_dirty); +spin_unlock (&dq_list_lock);return 1 ; }
@@ -451,12 +454,8 @@ int dquot_commit(struct dquot *dquot)struct quota_info *dqopt =sb_dqopt (dquot->dq_sb);mutex_lock (&dquot->dq_lock); -spin_lock (&dq_list_lock); -if (!clear_dquot_dirty (dquot)) { -spin_unlock (&dq_list_lock); +if (!clear_dquot_dirty (dquot)) goto out_lock; - } -spin_unlock (&dq_list_lock);/* Inactive dquot can be only if there was error during read/init * => we have better not writing it */ if (test_bit (DQ_ACTIVE_B , &dquot->dq_flags))
@@ -772,9 +771,7 @@ void dqput(struct dquot *dquot)* We clear dirty bit anyway, so that we avoid * infinite loop here */ -spin_lock (&dq_list_lock);clear_dquot_dirty (dquot); -spin_unlock (&dq_list_lock); } goto we_slept; }
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 20/27 quota: Remove dq_wait_unused from dquot
In reply to Jan KaraCurrently every dquot carries a wait_queue_head_t used only when we are turning quotas off to wait for last users to drop dquot references. Since such rare case is not performance sensitive in any means, just use a global waitqueue for this and save space in struct dquot. Also convert the logic to use wait_event() instead of open-coding it.
--- fs/quota/dquot.c | 17 +++++++---------- include/linux/quota.h | 1 - 2 files changed, 7 insertions(+), 11 deletions(-)
fs/quota/dquot.c +7 -10
@@ -126,6 +126,8 @@ __cacheline_aligned_in_smp DEFINE_SPINLOCK(dq_data_lock);EXPORT_SYMBOL (dq_data_lock);DEFINE_STATIC_SRCU (dquot_srcu); +static DECLARE_WAIT_QUEUE_HEAD (dquot_ref_wq); +void __quota_error (struct super_block *sb,const char *func,const char *fmt, ...) {
@@ -527,22 +529,18 @@ static void invalidate_dquots(struct super_block *sb, int type)continue ;/* Wait for dquot users */ if (atomic_read (&dquot->dq_count)) { -DEFINE_WAIT (wait); -dqgrab (dquot); -prepare_to_wait (&dquot->dq_wait_unused, &wait, -TASK_UNINTERRUPTIBLE );spin_unlock (&dq_list_lock); -/* Once dqput() wakes us up, we know it's time to free +/* +* Once dqput() wakes us up, we know it's time to free * the dquot. * IMPORTANT: we rely on the fact that there is always * at most one process waiting for dquot to free. * Otherwise dq_count would be > 1 and we would never * wake up. */ -if (atomic_read (&dquot->dq_count) >1 ) -schedule (); -finish_wait (&dquot->dq_wait_unused, &wait); +wait_event (dquot_ref_wq, +atomic_read (&dquot->dq_count) ==1 );dqput (dquot);/* At this moment dquot() need not exist (it could be * reclaimed by prune_dqcache(). Hence we must
@@ -754,7 +752,7 @@ void dqput(struct dquot *dquot)/* Releasing dquot during quotaoff phase? */ if (!sb_has_quota_active (dquot->dq_sb, dquot->dq_id.type) &&atomic_read (&dquot->dq_count) ==1 ) -wake_up (&dquot->dq_wait_unused); +wake_up (&dquot_ref_wq);spin_unlock (&dq_list_lock);return ; }
@@ -809,7 +807,6 @@ static struct dquot *get_empty_dquot(struct super_block *sb, int type)INIT_LIST_HEAD (&dquot->dq_inuse);INIT_HLIST_NODE (&dquot->dq_hash);INIT_LIST_HEAD (&dquot->dq_dirty); -init_waitqueue_head (&dquot->dq_wait_unused); dquot->dq_sb = sb; dquot->dq_id =make_kqid_invalid (type);atomic_set (&dquot->dq_count,1 );
include/linux/quota.h +0 -1
@@ -299,7 +299,6 @@ struct dquot {struct list_head dq_dirty;/* List of dirty dquots */ struct mutex dq_lock;/* dquot IO lock */ atomic_t dq_count;/* Use count */ -wait_queue_head_t dq_wait_unused;/* Wait queue for dquot to become unused */ struct super_block *dq_sb;/* superblock this applies to */ struct kqid dq_id;/* ID this applies to (uid, gid, projid) */ loff_t dq_off;/* Offset of dquot on disk */
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 21/27 quota: Allow disabling tracking of dirty dquots in a list
In reply to Jan KaraFilesystems that are journalling quotas generally don't need tracking of dirty dquots in a list since forcing a transaction commit flushes all quotas anyway. Allow filesystem to say it doesn't want dquots to be tracked as it reduces contention on the dq_list_lock.
--- fs/quota/dquot.c | 6 ++++++ include/linux/quota.h | 3 +++ 2 files changed, 9 insertions(+)
fs/quota/dquot.c +6 -0
@@ -344,6 +344,9 @@ int dquot_mark_dquot_dirty(struct dquot *dquot)if (!test_bit (DQ_ACTIVE_B , &dquot->dq_flags))return 0 ; +if (sb_dqopt (dquot->dq_sb)->flags &DQUOT_NOLIST_DIRTY ) +return test_and_set_bit (DQ_MOD_B , &dquot->dq_flags); +/* If quota is dirty already, we don't have to acquire dq_list_lock */ if (test_bit (DQ_MOD_B , &dquot->dq_flags))return 1 ;
@@ -385,6 +388,9 @@ static inline void dqput_all(struct dquot **dquot)static inline int clear_dquot_dirty (struct dquot *dquot) { +if (sb_dqopt (dquot->dq_sb)->flags &DQUOT_NOLIST_DIRTY ) +return test_and_clear_bit (DQ_MOD_B , &dquot->dq_flags); +spin_lock (&dq_list_lock);if (!test_and_clear_bit (DQ_MOD_B , &dquot->dq_flags)) {spin_unlock (&dq_list_lock);
include/linux/quota.h +3 -0
@@ -490,6 +490,9 @@ enum {*/ #define DQUOT_NEGATIVE_USAGE (1 << (DQUOT_STATE_LAST + 1))/* Allow negative quota usage */ +/* Do not track dirty dquots in a list */ +#define DQUOT_NOLIST_DIRTY (1 << (DQUOT_STATE_LAST + 2)) +static inline unsigned int dquot_state_flag (unsigned int flags,int type) {return flags << type;
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 22/27 ext4: Disable dirty list tracking of dquots when journalling quotas
In reply to Jan KaraWhen journalling quotas, we writeback all dquots immediately after changing them as part of current transation. Thus there's no need to write anything in dquot_writeback_dquots() and so we can avoid updating list of dirty dquots to reduce dq_list_lock contention.
This change reduces time to create 500000 files on ext4 on ramdisk by 50 different processes in separate directories by 15% when user quota is turned on.
--- fs/ext4/super.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
fs/ext4/super.c +8 -1
@@ -5404,6 +5404,13 @@ static int ext4_quota_on(struct super_block *sb, int type, int format_id,ext4_msg (sb,KERN_WARNING ,"Quota file not on filesystem root. " "Journaled quota will not work" ); +sb_dqopt (sb)->flags |=DQUOT_NOLIST_DIRTY ; + }else { +/* +* Clear the flag just in case mount options changed since +* last time. +*/ +sb_dqopt (sb)->flags &= ~DQUOT_NOLIST_DIRTY ; }/*
@@ -5500,7 +5507,7 @@ static int ext4_enable_quotas(struct super_block *sb)test_opt (sb,PRJQUOTA ), }; -sb_dqopt (sb)->flags |=DQUOT_QUOTA_SYS_FILE ; +sb_dqopt (sb)->flags |=DQUOT_QUOTA_SYS_FILE |DQUOT_NOLIST_DIRTY ;for (type =0 ; type <EXT4_MAXQUOTAS ; type++) {if (qf_inums[type]) { err = ext4_quota_enable(sb, type, QFMT_VFS_V1,
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 23/27 quota: Inline functions into their callsites
In reply to Jan Karainode_add_rsv_space() and inode_sub_rsv_space() had only one callsite. Inline them there directly. inode_claim_rsv_space() and inode_reclaim_rsv_space() had two callsites so inline them there as well. This will simplify further locking changes.
--- fs/quota/dquot.c | 72 +++++++++++++++++++----------------------------- include/linux/quotaops.h | 5 ---- 2 files changed, 28 insertions(+), 49 deletions(-)
fs/quota/dquot.c +28 -44
@@ -1583,40 +1583,6 @@ static qsize_t *inode_reserved_space(struct inode * inode)return inode->i_sb->dq_op->get_reserved_space (inode); } -void inode_add_rsv_space (struct inode *inode,qsize_t number) -{ -spin_lock (&inode->i_lock); - *inode_reserved_space (inode) += number; -spin_unlock (&inode->i_lock); -} -EXPORT_SYMBOL (inode_add_rsv_space); - -void inode_claim_rsv_space (struct inode *inode,qsize_t number) -{ -spin_lock (&inode->i_lock); - *inode_reserved_space (inode) -= number; -__inode_add_bytes (inode, number); -spin_unlock (&inode->i_lock); -} -EXPORT_SYMBOL (inode_claim_rsv_space); - -void inode_reclaim_rsv_space (struct inode *inode,qsize_t number) -{ -spin_lock (&inode->i_lock); - *inode_reserved_space (inode) += number; -__inode_sub_bytes (inode, number); -spin_unlock (&inode->i_lock); -} -EXPORT_SYMBOL (inode_reclaim_rsv_space); - -void inode_sub_rsv_space (struct inode *inode,qsize_t number) -{ -spin_lock (&inode->i_lock); - *inode_reserved_space (inode) -= number; -spin_unlock (&inode->i_lock); -} -EXPORT_SYMBOL (inode_sub_rsv_space); -static qsize_t inode_get_rsv_space (struct inode *inode) {qsize_t ret;
@@ -1632,18 +1598,24 @@ static qsize_t inode_get_rsv_space(struct inode *inode)static void inode_incr_space (struct inode *inode,qsize_t number,int reserve) { -if (reserve) -inode_add_rsv_space (inode, number); -else +if (reserve) { +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) += number; +spin_unlock (&inode->i_lock); + }else {inode_add_bytes (inode, number); + } }static void inode_decr_space (struct inode *inode,qsize_t number,int reserve) { -if (reserve) -inode_sub_rsv_space (inode, number); -else +if (reserve) { +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) -= number; +spin_unlock (&inode->i_lock); + }else {inode_sub_bytes (inode, number); + } }/*
@@ -1759,7 +1731,10 @@ int dquot_claim_space_nodirty(struct inode *inode, qsize_t number)int cnt, index;if (!dquot_active (inode)) { -inode_claim_rsv_space (inode, number); +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) -= number; +__inode_add_bytes (inode, number); +spin_unlock (&inode->i_lock);return 0 ; }
@@ -1772,7 +1747,10 @@ int dquot_claim_space_nodirty(struct inode *inode, qsize_t number)dquot_claim_reserved_space (dquots[cnt], number); }/* Update inode bytes */ -inode_claim_rsv_space (inode, number); +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) -= number; +__inode_add_bytes (inode, number); +spin_unlock (&inode->i_lock);spin_unlock (&dq_data_lock);mark_all_dquot_dirty (dquots);srcu_read_unlock (&dquot_srcu, index);
@@ -1789,7 +1767,10 @@ void dquot_reclaim_space_nodirty(struct inode *inode, qsize_t number)int cnt, index;if (!dquot_active (inode)) { -inode_reclaim_rsv_space (inode, number); +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) += number; +__inode_sub_bytes (inode, number); +spin_unlock (&inode->i_lock);return ; }
@@ -1802,7 +1783,10 @@ void dquot_reclaim_space_nodirty(struct inode *inode, qsize_t number)dquot_reclaim_reserved_space (dquots[cnt], number); }/* Update inode bytes */ -inode_reclaim_rsv_space (inode, number); +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) += number; +__inode_sub_bytes (inode, number); +spin_unlock (&inode->i_lock);spin_unlock (&dq_data_lock);mark_all_dquot_dirty (dquots);srcu_read_unlock (&dquot_srcu, index);
include/linux/quotaops.h +0 -5
@@ -38,11 +38,6 @@ void __quota_error(struct super_block *sb, const char *func,/* * declaration of quota_function calls in kernel. */ -void inode_add_rsv_space (struct inode *inode,qsize_t number); -void inode_claim_rsv_space (struct inode *inode,qsize_t number); -void inode_sub_rsv_space (struct inode *inode,qsize_t number); -void inode_reclaim_rsv_space (struct inode *inode,qsize_t number); -int dquot_initialize (struct inode *inode);bool dquot_initialize_needed (struct inode *inode);void dquot_drop (struct inode *inode);
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 24/27 quota: Inline inode_{incr,decr}_space() into callsites
In reply to Jan Karainode_incr_space() and inode_decr_space() have only two callsites. Inline them there as that will make locking changes simpler.
--- fs/quota/dquot.c | 55 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 28 insertions(+), 27 deletions(-)
fs/quota/dquot.c +28 -27
@@ -1595,29 +1595,6 @@ static qsize_t inode_get_rsv_space(struct inode *inode)return ret; } -static void inode_incr_space (struct inode *inode,qsize_t number, -int reserve) -{ -if (reserve) { -spin_lock (&inode->i_lock); - *inode_reserved_space (inode) += number; -spin_unlock (&inode->i_lock); - }else { -inode_add_bytes (inode, number); - } -} - -static void inode_decr_space (struct inode *inode,qsize_t number,int reserve) -{ -if (reserve) { -spin_lock (&inode->i_lock); - *inode_reserved_space (inode) -= number; -spin_unlock (&inode->i_lock); - }else { -inode_sub_bytes (inode, number); - } -} -/* * This functions updates i_blocks+i_bytes fields and quota information * (together with appropriate checks).
@@ -1639,7 +1616,13 @@ int __dquot_alloc_space(struct inode *inode, qsize_t number, int flags)struct dquot **dquots;if (!dquot_active (inode)) { -inode_incr_space (inode, number, reserve); +if (reserve) { +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) += number; +spin_unlock (&inode->i_lock); + }else { +inode_add_bytes (inode, number); + } goto out; }
@@ -1667,7 +1650,13 @@ int __dquot_alloc_space(struct inode *inode, qsize_t number, int flags)else dquot_incr_space (dquots[cnt], number); } -inode_incr_space (inode, number, reserve); +if (reserve) { +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) += number; +spin_unlock (&inode->i_lock); + }else { +inode_add_bytes (inode, number); + }spin_unlock (&dq_data_lock);if (reserve)
@@ -1805,7 +1794,13 @@ void __dquot_free_space(struct inode *inode, qsize_t number, int flags)int reserve = flags &DQUOT_SPACE_RESERVE , index;if (!dquot_active (inode)) { -inode_decr_space (inode, number, reserve); +if (reserve) { +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) -= number; +spin_unlock (&inode->i_lock); + }else { +inode_sub_bytes (inode, number); + }return ; }
@@ -1826,7 +1821,13 @@ void __dquot_free_space(struct inode *inode, qsize_t number, int flags)else dquot_decr_space (dquots[cnt], number); } -inode_decr_space (inode, number, reserve); +if (reserve) { +spin_lock (&inode->i_lock); + *inode_reserved_space (inode) -= number; +spin_unlock (&inode->i_lock); + }else { +inode_sub_bytes (inode, number); + }spin_unlock (&dq_data_lock);if (reserve)
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 25/27 quota: Inline dquot_[re]claim_reserved_space() into callsite
In reply to Jan Karadquot_claim_reserved_space() and dquot_reclaim_reserved_space() have only a single callsite. Inline them there.
--- fs/quota/dquot.c | 41 ++++++++++++++++------------------------- 1 file changed, 16 insertions(+), 25 deletions(-)
fs/quota/dquot.c +16 -25
@@ -1088,27 +1088,6 @@ static inline void dquot_resv_space(struct dquot *dquot, qsize_t number) dquot->dq_dqb.dqb_rsvspace += number; } -/* -* Claim reserved quota space -*/ -static void dquot_claim_reserved_space (struct dquot *dquot,qsize_t number) -{ -if (dquot->dq_dqb.dqb_rsvspace < number) { -WARN_ON_ONCE (1 ); - number = dquot->dq_dqb.dqb_rsvspace; - } - dquot->dq_dqb.dqb_curspace += number; - dquot->dq_dqb.dqb_rsvspace -= number; -} - -static void dquot_reclaim_reserved_space (struct dquot *dquot,qsize_t number) -{ -if (WARN_ON_ONCE (dquot->dq_dqb.dqb_curspace < number)) - number = dquot->dq_dqb.dqb_curspace; - dquot->dq_dqb.dqb_rsvspace += number; - dquot->dq_dqb.dqb_curspace -= number; -} -static inline void dquot_free_reserved_space (struct dquot *dquot,qsize_t number) {
@@ -1732,8 +1711,14 @@ int dquot_claim_space_nodirty(struct inode *inode, qsize_t number)spin_lock (&dq_data_lock);/* Claim reserved quotas to allocated quotas */ for (cnt =0 ; cnt <MAXQUOTAS ; cnt++) { -if (dquots[cnt]) -dquot_claim_reserved_space (dquots[cnt], number); +if (dquots[cnt]) { +struct dquot *dquot = dquots[cnt]; + +if (WARN_ON_ONCE (dquot->dq_dqb.dqb_rsvspace < number)) + number = dquot->dq_dqb.dqb_rsvspace; + dquot->dq_dqb.dqb_curspace += number; + dquot->dq_dqb.dqb_rsvspace -= number; + } }/* Update inode bytes */ spin_lock (&inode->i_lock);
@@ -1768,8 +1753,14 @@ void dquot_reclaim_space_nodirty(struct inode *inode, qsize_t number)spin_lock (&dq_data_lock);/* Claim reserved quotas to allocated quotas */ for (cnt =0 ; cnt <MAXQUOTAS ; cnt++) { -if (dquots[cnt]) -dquot_reclaim_reserved_space (dquots[cnt], number); +if (dquots[cnt]) { +struct dquot *dquot = dquots[cnt]; + +if (WARN_ON_ONCE (dquot->dq_dqb.dqb_curspace < number)) + number = dquot->dq_dqb.dqb_curspace; + dquot->dq_dqb.dqb_rsvspace += number; + dquot->dq_dqb.dqb_curspace -= number; + } }/* Update inode bytes */ spin_lock (&inode->i_lock);
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 26/27 fs: Provide __inode_get_bytes()
In reply to Jan KaraProvide helper __inode_get_bytes() which assumes i_lock is already acquired. Quota code will need this to be able to use i_lock to protect consistency of quota accounting information and inode usage.
--- fs/stat.c | 2 +- include/linux/fs.h | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-)
fs/stat.c +1 -1
@@ -710,7 +710,7 @@ loff_t inode_get_bytes(struct inode *inode)loff_t ret;spin_lock (&inode->i_lock); - ret = (((loff_t )inode->i_blocks) <<9 ) + inode->i_bytes; + ret =__inode_get_bytes (inode);spin_unlock (&inode->i_lock);return ret; }
include/linux/fs.h +4 -0
@@ -2998,6 +2998,10 @@ void __inode_add_bytes(struct inode *inode, loff_t bytes);void inode_add_bytes (struct inode *inode,loff_t bytes);void __inode_sub_bytes (struct inode *inode,loff_t bytes);void inode_sub_bytes (struct inode *inode,loff_t bytes); +static inline loff_t __inode_get_bytes (struct inode *inode) +{ +return (((loff_t )inode->i_blocks) <<9 ) + inode->i_bytes; +}loff_t inode_get_bytes (struct inode *inode);void inode_set_bytes (struct inode *inode,loff_t bytes); const char *simple_get_link(struct dentry *, struct inode *,
Signature · 1 line
2.12.3
Jan Kara
reviewerPATCH v2 27/27 quota: Reduce contention on dq_data_lock
In reply to Jan Karadq_data_lock is currently used to protect all modifications of quota accounting information, consistency of quota accounting on the inode, and dquot pointers from inode. As a result contention on the lock can be pretty heavy.
Reduce the contention on the lock by protecting quota accounting information by a new dquot->dq_dqb_lock and consistency of quota accounting with inode usage by inode->i_lock.
This change reduces time to create 500000 files on ext4 on ramdisk by 50 different processes in separate directories by 6% when user quota is turned on. When those 50 processes belong to 50 different users, the improvement is about 9%.
--- fs/ext4/super.c | 4 +- fs/ocfs2/quota_global.c | 8 +- fs/ocfs2/quota_local.c | 8 +- fs/quota/dquot.c | 287 ++++++++++++++++++++++++++++-------------------- fs/quota/quota_tree.c | 8 +- include/linux/quota.h | 3 +- 6 files changed, 186 insertions(+), 132 deletions(-)
fs/ext4/super.c +2 -2
@@ -5194,7 +5194,7 @@ static int ext4_statfs_project(struct super_block *sb, dquot =dqget (sb, qid);if (IS_ERR (dquot))return PTR_ERR (dquot); -spin_lock (&dq_data_lock); +spin_lock (&dquot->dq_dqb_lock); limit = (dquot->dq_dqb.dqb_bsoftlimit ? dquot->dq_dqb.dqb_bsoftlimit :
@@ -5217,7 +5217,7 @@ static int ext4_statfs_project(struct super_block *sb, (buf->f_files - dquot->dq_dqb.dqb_curinodes) : 0; } -spin_unlock (&dq_data_lock); +spin_unlock (&dquot->dq_dqb_lock);dqput (dquot);return 0 ; }
fs/ocfs2/quota_global.c +4 -4
@@ -504,7 +504,7 @@ int __ocfs2_sync_dquot(struct dquot *dquot, int freeing)/* Update space and inode usage. Get also other information from * global quota file so that we don't overwrite any changes there. * We are */ -spin_lock (&dq_data_lock); +spin_lock (&dquot->dq_dqb_lock); spacechange = dquot->dq_dqb.dqb_curspace -OCFS2_DQUOT (dquot)->dq_origspace; inodechange = dquot->dq_dqb.dqb_curinodes -
@@ -560,7 +560,7 @@ int __ocfs2_sync_dquot(struct dquot *dquot, int freeing)__clear_bit (DQ_LASTSET_B +QIF_ITIME_B , &dquot->dq_flags);OCFS2_DQUOT (dquot)->dq_origspace = dquot->dq_dqb.dqb_curspace;OCFS2_DQUOT (dquot)->dq_originodes = dquot->dq_dqb.dqb_curinodes; -spin_unlock (&dq_data_lock); +spin_unlock (&dquot->dq_dqb_lock); err =ocfs2_qinfo_lock (info, freeing);if (err <0 ) { mlog(ML_ERROR, "Failed to lock quota info, losing quota write"
@@ -924,10 +924,10 @@ static int ocfs2_mark_dquot_dirty(struct dquot *dquot)/* In case user set some limits, sync dquot immediately to global * quota file so that information propagates quicker */ -spin_lock (&dq_data_lock); +spin_lock (&dquot->dq_dqb_lock);if (dquot->dq_flags & mask) sync =1 ; -spin_unlock (&dq_data_lock); +spin_unlock (&dquot->dq_dqb_lock);/* This is a slight hack but we can't afford getting global quota * lock if we already have a transaction started. */ if (!sync ||journal_current_handle ()) {
fs/ocfs2/quota_local.c +4 -4
@@ -521,7 +521,7 @@ static int ocfs2_recover_local_quota_file(struct inode *lqinode, goto out_drop_lock; }down_write (&sb_dqopt (sb)->dqio_sem); -spin_lock (&dq_data_lock); +spin_lock (&dquot->dq_dqb_lock);/* Add usage from quota entry into quota changes * of our node. Auxiliary variables are important * due to signedness */
@@ -529,7 +529,7 @@ static int ocfs2_recover_local_quota_file(struct inode *lqinode, inodechange =le64_to_cpu (dqblk->dqb_inodemod); dquot->dq_dqb.dqb_curspace += spacechange; dquot->dq_dqb.dqb_curinodes += inodechange; -spin_unlock (&dq_data_lock); +spin_unlock (&dquot->dq_dqb_lock);/* We want to drop reference held by the crashed * node. Since we have our own reference we know * global structure actually won't be freed. */
@@ -877,12 +877,12 @@ static void olq_set_dquot(struct buffer_head *bh, void *private) dqblk->dqb_id =cpu_to_le64 (from_kqid (&init_user_ns, od->dq_dquot.dq_id)); -spin_lock (&dq_data_lock); +spin_lock (&od->dq_dquot.dq_dqb_lock); dqblk->dqb_spacemod =cpu_to_le64 (od->dq_dquot.dq_dqb.dqb_curspace - od->dq_origspace); dqblk->dqb_inodemod =cpu_to_le64 (od->dq_dquot.dq_dqb.dqb_curinodes - od->dq_originodes); -spin_unlock (&dq_data_lock); +spin_unlock (&od->dq_dquot.dq_dqb_lock); trace_olq_set_dquot( (unsigned long long)le64_to_cpu(dqblk->dqb_spacemod), (unsigned long long)le64_to_cpu(dqblk->dqb_inodemod),
fs/quota/dquot.c +170 -117
574 lines not on this page · raw message
fs/quota/quota_tree.c +4 -4
@@ -389,9 +389,9 @@ int qtree_write_dquot(struct qtree_mem_dqinfo *info, struct dquot *dquot)return ret; } } -spin_lock (&dq_data_lock); +spin_lock (&dquot->dq_dqb_lock); info->dqi_ops->mem2disk_dqblk (ddquot, dquot); -spin_unlock (&dq_data_lock); +spin_unlock (&dquot->dq_dqb_lock); ret = sb->s_op->quota_write (sb, type, ddquot, info->dqi_entry_size, dquot->dq_off);if (ret != info->dqi_entry_size) {
@@ -649,14 +649,14 @@ int qtree_read_dquot(struct qtree_mem_dqinfo *info, struct dquot *dquot)kfree (ddquot); goto out; } -spin_lock (&dq_data_lock); +spin_lock (&dquot->dq_dqb_lock); info->dqi_ops->disk2mem_dqblk (dquot, ddquot);if (!dquot->dq_dqb.dqb_bhardlimit && !dquot->dq_dqb.dqb_bsoftlimit && !dquot->dq_dqb.dqb_ihardlimit && !dquot->dq_dqb.dqb_isoftlimit)set_bit (DQ_FAKE_B , &dquot->dq_flags); -spin_unlock (&dq_data_lock); +spin_unlock (&dquot->dq_dqb_lock);kfree (ddquot); out:dqstats_inc (DQST_READS );
include/linux/quota.h +2 -1
@@ -298,12 +298,13 @@ struct dquot {struct list_head dq_free;/* Free list element */ struct list_head dq_dirty;/* List of dirty dquots */ struct mutex dq_lock;/* dquot IO lock */ +spinlock_t dq_dqb_lock;/* Lock protecting dq_dqb changes */ atomic_t dq_count;/* Use count */ struct super_block *dq_sb;/* superblock this applies to */ struct kqid dq_id;/* ID this applies to (uid, gid, projid) */ loff_t dq_off;/* Offset of dquot on disk */ unsigned long dq_flags;/* See DQ_* */ -struct mem_dqblk dq_dqb;/* Diskquota usage */ +struct mem_dqblk dq_dqb;/* Diskquota usage [dq_dqb_lock] */ };/* Operations which must be implemented by each quota format */
Signature · 1 line
2.12.3
A message that is not in the archive (sent off-list or not yet received): CAP9B-Qnnh3V9kq1qHemWD5s7DmA3Ef3LAVvgzLrbb7pgLEqb8w@mail.gmail.com
Jan Kara
reviewerQuoted 26 lines · Wang Shilong
On Mon, Aug 21, 2017 at 7:54 PM, Jan Kara <[email protected]> wrote:
� Hello,
this is a series of patches that address scalability issues in quota subsystem.
For my testing I've used a synthetic benchmark where 50 different processes are
creating files 10000 files each in separate directories (either for different
or the same user). I've been testing on a machine with 48 CPUs with 64GB of RAM over either SATA drive or ramdisk. Different patches have shown improvements in different configs (see details in the changelog). Also Wang Shilong gave some testing to my patches and in his setup he saw about 100% improvement for file creation and about 50% improvement for file unlink.We rebechmark again this latest patchset, generally in our test case. with file creation improved patch together with/without this quota patchset, result is:
(quoted diff) +0
-2-O quota +30% for creation, +10% for unlink -O quota, porject +43% for creation, +67% for unlink
Quoted 1 line
Almost no degradation with/without quota for unlink now.
Great, thanks for the data! BTW, how big difference do you observe with / without quota for creates?
Quoted 2 lines
Jan Kara, just for confirm will these patches be sent to Linus once 4.14 merge window open?
Yes, patches are queued in my tree for the coming merge window.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
A message that is not in the archive (sent off-list or not yet received): CAP9B-QntdC06C8fAinqU6-4V6SzC_k4SyEpgHAqZ5zEF7ERq-w@mail.gmail.com
Jan Kara
reviewerQuoted 7 lines · Wang Shilong
This is without your patch:
no Quota � � �quota � � quota,project � �� 1 �851,207 � � 341,941 � 178,686 � �� 2 �850,368 � � 342,233 � 191,755 � �� 3 �848,877 � � 342,768 � �193,807
Quoted 7 lines
With your patchset:
no quota � � �quota � � �quota,project 1 �853,391 � 448,378 �385,292�
2 �851,379 � 448,375 �407,716 �� 3 �850,203 � 448,415 �406,813 ��We still see creation regression here, but your patchset help a lot.
OK, still quite some way to go... There will always be some cost associated with quota bookkeeping especially since this has to be synchronized across CPUs to allow for reliable limits checking. But 50% seems quite a bit.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
A message that is not in the archive (sent off-list or not yet received): CAP9B-Qn1VvfnDZNH4eM1rWxenO6M=KJ+mSurEuc2FC70YwWoyQ@mail.gmail.com
Jan Kara
reviewerQuoted 25 lines · Wang Shilong
Hello,
On Thu, Aug 31, 2017 at 5:48 PM, Jan Kara <[email protected]> wrote:
On Thu 31-08-17 17:09:26, Wang Shilong wrote: > This is without your patch: > > no Quota � � �quota � � quota,project � �� > 1 �851,207 � � 341,941 � 178,686 � �� > 2 �850,368 � � 342,233 � 191,755 � �� > 3 �848,877 � � 342,768 � �193,807 > > With your patchset: > no quota � � �quota � � �quota,project > 1 �853,391 � 448,378 �385,292� > 2 �851,379 � 448,375 �407,716 �� > 3 �850,203 � 448,415 �406,813 �� > > We still see creation regression here, but your patchset help a lot. OK, still quite some way to go... There will always be some cost associated with quota bookkeeping especially since this has to be synchronized across CPUs to allow for reliable limits checking. But 50% seems quite a bit.
Quoted 4 lines
We run tests with quota enabled, and collecting lock statics using 'perf lock'
Check IO/CPU overload!
Actually, this doesn't look bad. Only 1626 contentions on s_inode_list_lock, and we totally waited only for 5ms over all CPUs. So I'm pretty confident contention on that lock is not costing us much. 'perf lock' seems to show only spinlocks while I think the real contention is going to be on sleeping locks. You should be able to see that using lockstat framework in the kernel (Documentation/locking/lockstat.txt).
Honza
Quoted 48 lines
�Name � acquired �contended � avg wait (ns) total wait (ns) � max wait (ns) � min wait (ns)�
�&(&s->s_inode_li... � � �16850 � � � 1626 � � � � � �3060 � � � � 4976891 � � � � � 77915 � � � � � �1297�
�&(&lru->node[i].... � � � 8332 � � � �434 � � � � � �3506 � � � � 1521648 � � � � �317040 � � � � � �1227�
�&(&journal->j_li... � � � 3724 � � � �105 � � � � � �4784 � � � � �502326 � � � � � 64196 � � � � � �1311�
� � �sparse_irq_lock � � � 3248 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�
�&(&lock->wait_lo... � � � 2636 � � � �144 � � � � � 10178 � � � � 1465730 � � � � �167087 � � � � � �1345�
�&(&lock->wait_lo... � � � 2184 � � � � 81 � � � � � 11999 � � � � �971986 � � � � �272372 � � � � � �1369�
�&(&zone->lock)->... � � � 2010 � � � � 51 � � � � � �2080 � � � � �106127 � � � � � �4945 � � � � � � 876�
�&(&dentry->d_loc... � � � 1945 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�
�&(&dentry->d_loc... � � � 1934 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�
�&(&dentry->d_loc... � � � 1933 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�
�&(&dentry->d_loc... � � � 1918 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�
�&(&dentry->d_loc... � � � 1915 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�
�&(&dentry->d_loc... � � � 1910 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�
�&(&dentry->d_loc... � � � 1906 � � � � �0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0 � � � � � � � 0�It looks we are still contending on @s_inode_list_lock which related to quota.
Thanks,
Shilong� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR