exfat: take s_lock in read mode for iomap mapping paths
cover exfat: take s_lock in read mode for iomap mapping paths
From: Chi Zhiling <[email protected]>
The iomap mapping paths currently serialize all inode mappings on the sbi->s_lock mutex. This series converts it to an rw_semaphore and takes it in read mode on the mapping paths so that mappings of distinct inodes proceed in parallel. The preceding patches make the paths safe to run
concurrently: bitmap_lock covers the allocation state, atomic bit ops cover the volume dirty flag, and the FAT2 mirror copy is serialized against writeback.
Same-inode access stays serialized by the exclusive inode_lock; writeback does not hold it but is safe because truncate flushes and truncates the page cache first and cannot run concurrently with writeback. The lock ordering (inode_lock -> s_lock -> bitmap_lock) is unchanged, so no new deadlock scenarios are introduced.
A per-inode read-write lock was also considered, but its implementation turned out to be considerably more complex, so this series relaxes the existing s_lock instead. Any suggestions on that approach are welcome.
The series passes the xfstests exfat suite. In Unixbench, the fstime-w score improved from 3977196.8 to 4746349.6 (~+19%). Any comments and suggestions are welcome.
Chi Zhiling (6):
exfat: remove dead hint_bmap updates in I/O and truncate paths
exfat: take bitmap_lock at the start of exfat_alloc_cluster()
exfat: use atomic bit ops for volume dirty flag
exfat: lock FAT2 buffer while copying mirrored FAT entries
exfat: convert s_lock mutex to rw_semaphore using write lock
exfat: take s_lock in read mode for iomap mapping paths
fs/exfat/dir.c | 14 +++++++------- fs/exfat/exfat_fs.h | 5 ++--- fs/exfat/exfat_raw.h | 4 ++-- fs/exfat/fatent.c | 20 +++++++++++++------- fs/exfat/file.c | 8 ++------ fs/exfat/inode.c | 15 +++++---------- fs/exfat/iomap.c | 4 ++-- fs/exfat/namei.c | 26 +++++++++++++------------- fs/exfat/super.c | 42 +++++++++++++++++++++--------------------- 9 files changed, 67 insertions(+), 71 deletions(-)
Signature · 1 line
2.53.0
RFC PATCH 1/6 exfat: remove dead hint_bmap updates in I/O and truncate paths
From: Chi Zhiling <[email protected]>
hint_bmap is only consumed by exfat_readdir() during directory iteration. exfat_map_cluster() runs on the file I/O path (via iomap) and never reaches directory iteration, and __exfat_truncate() only resets the hint when the inode is already being evicted, so drop them along with the now-unused local_clu_offset.
--- fs/exfat/file.c | 4 ---- fs/exfat/inode.c | 5 ----- 2 files changed, 9 deletions(-)
fs/exfat/file.c +0 -4
@@ -278,10 +278,6 @@ int __exfat_truncate(struct inode *inode)/* clear exfat cache */ exfat_cache_inval_inode (inode); -/* hint information */ - ei->hint_bmap.off =EXFAT_EOF_CLUSTER ; - ei->hint_bmap.clu =EXFAT_EOF_CLUSTER ; -/* hint_stat will be used if this is directory. */ ei->hint_stat.eidx =0 ; ei->hint_stat.clu = ei->start_clu;
fs/exfat/inode.c +0 -5
@@ -144,7 +144,6 @@ int exfat_map_cluster(struct inode *inode, unsigned int clu_offset,struct super_block *sb = inode->i_sb;struct exfat_sb_info *sbi =EXFAT_SB (sb);struct exfat_inode_info *ei =EXFAT_I (inode); -unsigned int local_clu_offset = clu_offset;unsigned int num_to_be_allocated =0 , num_clusters; num_clusters =exfat_bytes_to_cluster (sbi,exfat_ondisk_size (inode));
@@ -234,10 +233,6 @@ int exfat_map_cluster(struct inode *inode, unsigned int clu_offset, *balloc = true; } -/* hint information */ - ei->hint_bmap.off = local_clu_offset; - ei->hint_bmap.clu = *clu; -return 0 ; }
Signature · 1 line
2.53.0
RFC PATCH 2/6 exfat: take bitmap_lock at the start of exfat_alloc_cluster()
In reply to Chi ZhilingFrom: Chi Zhiling <[email protected]>
Acquire sbi->bitmap_lock at the top of exfat_alloc_cluster() so the used_clusters/clu_srch_ptr validation and updates are covered by the lock, removing a window where the early checks raced with concurrent allocations.
--- fs/exfat/fatent.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
fs/exfat/fatent.c +10 -7
@@ -427,19 +427,22 @@ int exfat_alloc_cluster(struct inode *inode, unsigned int num_alloc,struct super_block *sb = inode->i_sb;struct exfat_sb_info *sbi =EXFAT_SB (sb); +mutex_lock (&sbi->bitmap_lock); + total_cnt =EXFAT_DATA_CLUSTER_COUNT (sbi);if (unlikely (total_cnt < sbi->used_clusters)) {exfat_fs_error_ratelimit (sb,"%s: invalid used clusters(t:%u,u:%u)\n" , __func__, total_cnt, sbi->used_clusters); -return -EIO ; + ret = -EIO ; + goto unlock; } -if (num_alloc > total_cnt - sbi->used_clusters) -return -ENOSPC ; - -mutex_lock (&sbi->bitmap_lock); +if (num_alloc > total_cnt - sbi->used_clusters) { + ret = -ENOSPC ; + goto unlock; + } hint_clu = p_chain->dir;/* find new cluster */
@@ -516,8 +519,8 @@ int exfat_alloc_cluster(struct inode *inode, unsigned int num_alloc, done: sbi->clu_srch_ptr = hint_clu; sbi->used_clusters += p_chain->size; -mutex_unlock (&sbi->bitmap_lock); -return 0 ; + ret =0 ; + goto unlock; } hint_clu = new_clu +1 ;
Signature · 1 line
2.53.0
RFC PATCH 3/6 exfat: use atomic bit ops for volume dirty flag
In reply to Chi ZhilingFrom: Chi Zhiling <[email protected]>
After converting s_lock to a reader-writer lock, multiple operations can execute concurrently under the read lock. As a result, multiple threads may concurrently update sbi->vol_flags, causing a race in the existing read-modify-write sequence.
Convert sbi->vol_flags to an unsigned long bitmap and use test_and_set_bit() / test_and_clear_bit() to atomically update the volume dirty bit. Only the thread performing the 0 -> 1 transition updates the boot sector, avoiding concurrent unsynchronized modifications of vol_flags and the boot-sector buffer.
Remove vol_flags_persistent, since the bitmap can directly retain the MEDIA_FAILURE state while preserving the complete 16-bit on-disk volume flags field when writing it back.
--- fs/exfat/exfat_fs.h | 3 +-- fs/exfat/exfat_raw.h | 4 ++-- fs/exfat/super.c | 28 ++++++++++++---------------- 3 files changed, 15 insertions(+), 20 deletions(-)
fs/exfat/exfat_fs.h +1 -2
@@ -232,8 +232,7 @@ struct exfat_sb_info {unsigned int num_FAT_sectors;/* num of FAT sectors */ unsigned int root_dir;/* root dir cluster */ unsigned int dentries_per_clu;/* num of dentries per cluster */ -unsigned int vol_flags;/* volume flags */ -unsigned int vol_flags_persistent;/* volume flags to retain */ +unsigned long vol_flags;/* volume flags (bitmap) */ struct buffer_head *boot_bh;/* buffer_head of BOOT sector */ unsigned int map_clu;/* allocation bitmap start cluster */
fs/exfat/exfat_raw.h +2 -2
@@ -14,8 +14,8 @@#define EXFAT_MAX_FILE_LEN 255 -#define VOLUME_DIRTY 0x0002 -#define MEDIA_FAILURE 0x0004 +#define VOLUME_DIRTY_BIT 1 +#define MEDIA_FAILURE_BIT 2#define EXFAT_EOF_CLUSTER 0xFFFFFFFFu#define EXFAT_BAD_CLUSTER 0xFFFFFFF7u
fs/exfat/super.c +12 -16
@@ -69,27 +69,18 @@ static int exfat_statfs(struct dentry *dentry, struct kstatfs *buf)return 0 ; } -static int exfat_set_vol_flags (struct super_block *sb,unsigned short new_flags) +static int exfat_sync_vol_flags (struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb);struct boot_sector *p_boot = (struct boot_sector *)sbi->boot_bh->b_data; -/* retain persistent-flags */ - new_flags |= sbi->vol_flags_persistent; - -/* flags are not changed */ -if (sbi->vol_flags == new_flags) -return 0 ; - - sbi->vol_flags = new_flags; -/* skip updating volume dirty flag, * if this volume has been mounted with read-only */ if (sb_rdonly (sb))return 0 ; - p_boot->vol_flags =cpu_to_le16 (new_flags); + p_boot->vol_flags =cpu_to_le16 ((unsigned short )READ_ONCE (sbi->vol_flags));set_buffer_uptodate (sbi->boot_bh);mark_buffer_dirty (sbi->boot_bh);
@@ -103,14 +94,20 @@ int exfat_set_volume_dirty(struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb); -return exfat_set_vol_flags (sb, sbi->vol_flags |VOLUME_DIRTY ); +if (test_and_set_bit (VOLUME_DIRTY_BIT , &sbi->vol_flags)) +return 0 ; + +return exfat_sync_vol_flags (sb); }int exfat_clear_volume_dirty (struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb); -return exfat_set_vol_flags (sb, sbi->vol_flags & ~VOLUME_DIRTY ); +if (!test_and_clear_bit (VOLUME_DIRTY_BIT , &sbi->vol_flags)) +return 0 ; + +return exfat_sync_vol_flags (sb); }static int exfat_show_options (struct seq_file *m,struct dentry *root)
@@ -509,7 +506,6 @@ static int exfat_read_boot_sector(struct super_block *sb) (sbi->cluster_size_bits -DENTRY_SIZE_BITS ); sbi->vol_flags =le16_to_cpu (p_boot->vol_flags); - sbi->vol_flags_persistent = sbi->vol_flags & (VOLUME_DIRTY |MEDIA_FAILURE ); sbi->clu_srch_ptr =EXFAT_FIRST_CLUSTER ;/* check consistencies */
@@ -526,9 +522,9 @@ static int exfat_read_boot_sector(struct super_block *sb)return -EINVAL ; } -if (sbi->vol_flags &VOLUME_DIRTY ) +if (test_bit (VOLUME_DIRTY_BIT , &sbi->vol_flags))exfat_warn (sb,"Volume was not properly unmounted. Some data may be corrupt. Please run fsck." ); -if (sbi->vol_flags &MEDIA_FAILURE ) +if (test_bit (MEDIA_FAILURE_BIT , &sbi->vol_flags))exfat_warn (sb,"Medium has reported failures. Some data may be lost." );/*
Signature · 1 line
2.53.0
Namjae Jeon
maintainerRe: [RFC PATCH v1 3/6] exfat: use atomic bit ops for volume dirty flag
fs/exfat/super.c static int exfat_statfs(struct dentry *dentry, struct kstatfs *buf) · line 78
Quoted 8 lines
fs/exfat/super.c +1
-3-static int exfat_set_vol_flags (struct super_block *sb,unsigned short new_flags) +static int exfat_sync_vol_flags (struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb);struct boot_sector *p_boot = (struct boot_sector *)sbi->boot_bh->b_data; -/* retain persistent-flags */ - new_flags |= sbi->vol_flags_persistent;
Please don't remove ->vol_flags_persistent. It prevents pre-existing volume dirty flags from being lost, but removing it clears a dirty flag set before mount during normal unmount.
Re: [RFC PATCH v1 3/6] exfat: use atomic bit ops for volume dirty flag
Hi, Namjae
You are right. This should not be removed. I will restore it in the next version.
Thanks,
Quoted 11 lines · Namjae Jeon
fs/exfat/super.c static int exfat_statfs(struct dentry *dentry, struct kstatfs *buf) · line 78
Quoted 8 lines
fs/exfat/super.c +1
-3-static int exfat_set_vol_flags (struct super_block *sb,unsigned short new_flags) +static int exfat_sync_vol_flags (struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb);struct boot_sector *p_boot = (struct boot_sector *)sbi->boot_bh->b_data; -/* retain persistent-flags */ - new_flags |= sbi->vol_flags_persistent;Please don't remove ->vol_flags_persistent. It prevents pre-existing volume dirty flags from being lost, but removing it clears a dirty flag set before mount during normal unmount.
RFC PATCH 4/6 exfat: lock FAT2 buffer while copying mirrored FAT entries
In reply to Chi ZhilingFrom: Chi Zhiling <[email protected]>
exfat_mirror_bh() updates the FAT2 buffer without serializing the copy against writeback. Once s_lock is converted to a read-write lock, multiple read-side operations may run concurrently with writeback. This can allow writeback to observe a partially updated FAT2 block.
Lock the destination buffer while copying the FAT entry data to serialize the update with writeback and prevent torn FAT2 writes.
--- fs/exfat/fatent.c | 3 +++ 1 file changed, 3 insertions(+)
fs/exfat/fatent.c +3 -0
@@ -24,7 +24,10 @@ static int exfat_mirror_bh(struct super_block *sb, struct buffer_head *bh) c_bh =sb_getblk (sb, sec2);if (!c_bh)return -ENOMEM ; +/* Serialize the copy with writeback to avoid a torn FAT2 write */ +lock_buffer (c_bh);memcpy (c_bh->b_data, bh->b_data, sb->s_blocksize); +unlock_buffer (c_bh); err =exfat_update_bh (c_bh, sb->s_flags &SB_SYNCHRONOUS );brelse (c_bh); }
Signature · 1 line
2.53.0
RFC PATCH 5/6 exfat: convert s_lock mutex to rw_semaphore using write lock
In reply to Chi ZhilingFrom: Chi Zhiling <[email protected]>
Convert sbi->s_lock from a mutex to an rw_semaphore and acquire it in write mode at every existing use site. This is a mechanical conversion with no behavioral change, preparing the lock for read-mode access in the iomap mapping paths so that concurrent mappings of distinct inodes can proceed in parallel.
Lock ordering stays consistent (inode_lock -> s_lock -> bitmap_lock), so no deadlock is introduced.
--- fs/exfat/dir.c | 14 +++++++------- fs/exfat/exfat_fs.h | 2 +- fs/exfat/file.c | 4 ++-- fs/exfat/inode.c | 10 +++++----- fs/exfat/iomap.c | 4 ++-- fs/exfat/namei.c | 26 +++++++++++++------------- fs/exfat/super.c | 10 +++++----- 7 files changed, 35 insertions(+), 35 deletions(-)
fs/exfat/dir.c +7 -7
@@ -223,7 +223,7 @@ static int exfat_iterate(struct file *file, struct dir_context *ctx)if (err) goto out; get_new: -mutex_lock (&EXFAT_SB (sb)->s_lock); +down_write (&EXFAT_SB (sb)->s_lock);if (ei->flags ==ALLOC_NO_FAT_CHAIN && cpos >=i_size_read (inode)) goto end_of_dir;
@@ -255,7 +255,7 @@ static int exfat_iterate(struct file *file, struct dir_context *ctx) inum =iunique (sb,EXFAT_ROOT_INO ); } -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock);if (!dir_emit (ctx, nb->lfn,strlen (nb->lfn), inum, (de.attr &EXFAT_ATTR_SUBDIR ) ?DT_DIR :DT_REG )) goto out;
@@ -266,7 +266,7 @@ static int exfat_iterate(struct file *file, struct dir_context *ctx)if (!cpos && fake_offset) cpos =ITER_POS_FILLED_DOTS ; ctx->pos = cpos; -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock); out:/* * To improve performance, free namebuf after unlock sb_lock.
@@ -1294,7 +1294,7 @@ int exfat_read_volume_label(struct super_block *sb, struct exfat_uni_name *labelstruct exfat_entry_set_cache es;struct exfat_dentry *ep; -mutex_lock (&sbi->s_lock); +down_write (&sbi->s_lock);memset (label_out,0 ,sizeof (*label_out)); ret =exfat_get_volume_label_dentry (sb, &es);
@@ -1322,7 +1322,7 @@ int exfat_read_volume_label(struct super_block *sb, struct exfat_uni_name *labelexfat_put_dentry_set (&es, false); unlock: -mutex_unlock (&sbi->s_lock); +up_write (&sbi->s_lock);return ret; }
@@ -1339,7 +1339,7 @@ int exfat_write_volume_label(struct super_block *sb,if (label->name_len >EXFAT_VOLUME_LABEL_LEN )return -EINVAL ; -mutex_lock (&sbi->s_lock); +down_write (&sbi->s_lock); ret =exfat_get_volume_label_dentry (sb, &es);if (ret == -ENOENT ) {
@@ -1376,6 +1376,6 @@ int exfat_write_volume_label(struct super_block *sb, ret =exfat_put_dentry_set (&es,IS_DIRSYNC (root_inode)); unlock: -mutex_unlock (&sbi->s_lock); +up_write (&sbi->s_lock);return ret; }
fs/exfat/exfat_fs.h +1 -1
@@ -246,7 +246,7 @@ struct exfat_sb_info {unsigned long s_exfat_flags;/* Exfat superblock flags */ -struct mutex s_lock;/* superblock lock */ +struct rw_semaphore s_lock;/* superblock lock */ struct mutex bitmap_lock;/* bitmap lock */ struct exfat_mount_options options;struct nls_table *nls_io;/* Charset used for input and display */
fs/exfat/file.c +2 -2
@@ -297,7 +297,7 @@ static void exfat_truncate(struct inode *inode)struct exfat_inode_info *ei =EXFAT_I (inode);int err; -mutex_lock (&sbi->s_lock); +down_write (&sbi->s_lock);if (ei->start_clu ==0 ) {/* * Empty start_clu != ~0 (not allocated)
@@ -312,7 +312,7 @@ static void exfat_truncate(struct inode *inode) inode->i_blocks =round_up (i_size_read (inode), sbi->cluster_size) >>9 ; write_size: -mutex_unlock (&sbi->s_lock); +up_write (&sbi->s_lock); } int exfat_getattr(struct mnt_idmap *idmap, const struct path *path,
fs/exfat/inode.c +5 -5
@@ -116,16 +116,16 @@ int exfat_write_inode(struct inode *inode, struct writeback_control *wbc)if (unlikely (exfat_forced_shutdown (inode->i_sb)))return -EIO ; -mutex_lock (&EXFAT_SB (inode->i_sb)->s_lock); +down_write (&EXFAT_SB (inode->i_sb)->s_lock); ret =__exfat_write_inode (inode, wbc->sync_mode ==WB_SYNC_ALL ); -mutex_unlock (&EXFAT_SB (inode->i_sb)->s_lock); +up_write (&EXFAT_SB (inode->i_sb)->s_lock);return ret; }void exfat_sync_inode (struct inode *inode) { -lockdep_assert_held (&EXFAT_SB (inode->i_sb)->s_lock); +lockdep_assert_held_write (&EXFAT_SB (inode->i_sb)->s_lock);__exfat_write_inode (inode,1 ); }
@@ -443,9 +443,9 @@ void exfat_evict_inode(struct inode *inode)if (!inode->i_nlink) {i_size_write (inode,0 ); -mutex_lock (&EXFAT_SB (inode->i_sb)->s_lock); +down_write (&EXFAT_SB (inode->i_sb)->s_lock);__exfat_truncate (inode); -mutex_unlock (&EXFAT_SB (inode->i_sb)->s_lock); +up_write (&EXFAT_SB (inode->i_sb)->s_lock); }clear_inode (inode);
fs/exfat/iomap.c +2 -2
@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length num_clusters =exfat_bytes_to_cluster_round_up (sbi, offset + length) -exfat_bytes_to_cluster (sbi, offset); -mutex_lock (&sbi->s_lock); +down_write (&sbi->s_lock); iomap->bdev = inode->i_sb->s_bdev; iomap->offset = offset;
@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length iomap->flags |=IOMAP_F_MERGED ; out: -mutex_unlock (&sbi->s_lock); +up_write (&sbi->s_lock);return err; }
fs/exfat/namei.c +13 -13
@@ -565,7 +565,7 @@ static int exfat_create(struct mnt_idmap *idmap, struct inode *dir,if (unlikely (exfat_forced_shutdown (sb)))return -EIO ; -mutex_lock (&EXFAT_SB (sb)->s_lock); +down_write (&EXFAT_SB (sb)->s_lock);exfat_set_volume_dirty (sb); err =exfat_add_entry (dir, dentry->d_name.name,TYPE_FILE , &info);if (err)
@@ -592,7 +592,7 @@ static int exfat_create(struct mnt_idmap *idmap, struct inode *dir,d_instantiate (dentry, inode); unlock: -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock);return err; }
@@ -732,7 +732,7 @@ static struct dentry *exfat_lookup(struct inode *dir, struct dentry *dentry,int err;loff_t i_pos; -mutex_lock (&EXFAT_SB (sb)->s_lock); +down_write (&EXFAT_SB (sb)->s_lock); err =exfat_find (dir, &dentry->d_name, &info);if (err) {if (unlikely (err != -ENOENT ))
@@ -761,12 +761,12 @@ static struct dentry *exfat_lookup(struct inode *dir, struct dentry *dentry,*/ d_move (alias, dentry);iput (inode); -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock);return alias; }dput (alias); out: -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock);if (!inode)exfat_d_version_set (dentry,inode_query_iversion (dir));
@@ -785,7 +785,7 @@ static int exfat_unlink(struct inode *dir, struct dentry *dentry)if (unlikely (exfat_forced_shutdown (sb)))return -EIO ; -mutex_lock (&EXFAT_SB (sb)->s_lock); +down_write (&EXFAT_SB (sb)->s_lock);if (ei->dir.dir ==DIR_DELETED ) {exfat_err (sb,"abnormal access to deleted dentry" ); err = -ENOENT ;
@@ -821,7 +821,7 @@ static int exfat_unlink(struct inode *dir, struct dentry *dentry)exfat_unhash_inode (inode);exfat_d_version_set (dentry,inode_query_iversion (dir)); unlock: -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock);return err; }
@@ -838,7 +838,7 @@ static struct dentry *exfat_mkdir(struct mnt_idmap *idmap, struct inode *dir,if (unlikely (exfat_forced_shutdown (sb)))return ERR_PTR (-EIO ); -mutex_lock (&EXFAT_SB (sb)->s_lock); +down_write (&EXFAT_SB (sb)->s_lock);exfat_set_volume_dirty (sb); err =exfat_add_entry (dir, dentry->d_name.name,TYPE_DIR , &info);if (err)
@@ -867,7 +867,7 @@ static struct dentry *exfat_mkdir(struct mnt_idmap *idmap, struct inode *dir,d_instantiate (dentry, inode); unlock: -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock);return err ?ERR_PTR (err) :NULL ; }
@@ -929,7 +929,7 @@ static int exfat_rmdir(struct inode *dir, struct dentry *dentry)if (unlikely (exfat_forced_shutdown (sb)))return -EIO ; -mutex_lock (&EXFAT_SB (inode->i_sb)->s_lock); +down_write (&EXFAT_SB (inode->i_sb)->s_lock);if (ei->dir.dir ==DIR_DELETED ) {exfat_err (sb,"abnormal access to deleted dentry" );
@@ -979,7 +979,7 @@ static int exfat_rmdir(struct inode *dir, struct dentry *dentry)exfat_unhash_inode (inode);exfat_d_version_set (dentry,inode_query_iversion (dir)); unlock: -mutex_unlock (&EXFAT_SB (inode->i_sb)->s_lock); +up_write (&EXFAT_SB (inode->i_sb)->s_lock);return err; }
@@ -1282,7 +1282,7 @@ static int exfat_rename(struct mnt_idmap *idmap,if (flags & ~RENAME_NOREPLACE )return -EINVAL ; -mutex_lock (&EXFAT_SB (sb)->s_lock); +down_write (&EXFAT_SB (sb)->s_lock); old_inode = old_dentry->d_inode; new_inode = new_dentry->d_inode;
@@ -1334,7 +1334,7 @@ static int exfat_rename(struct mnt_idmap *idmap, } unlock: -mutex_unlock (&EXFAT_SB (sb)->s_lock); +up_write (&EXFAT_SB (sb)->s_lock);return err; }
fs/exfat/super.c +5 -5
@@ -45,11 +45,11 @@ static void exfat_put_super(struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb); -mutex_lock (&sbi->s_lock); +down_write (&sbi->s_lock);exfat_clear_volume_dirty (sb);exfat_free_bitmap (sbi);brelse (sbi->boot_bh); -mutex_unlock (&sbi->s_lock); +up_write (&sbi->s_lock); }static int exfat_statfs (struct dentry *dentry,struct kstatfs *buf)
@@ -771,9 +771,9 @@ static int exfat_reconfigure(struct fs_context *fc) fc->sb_flags |=SB_NODIRATIME ;sync_filesystem (sb); -mutex_lock (&sbi->s_lock); +down_write (&sbi->s_lock);exfat_clear_volume_dirty (sb); -mutex_unlock (&sbi->s_lock); +up_write (&sbi->s_lock);if (new_opts->allow_utime == (unsigned short )-1 ) new_opts->allow_utime = ~new_opts->fs_dmask &0022 ;
@@ -820,7 +820,7 @@ static int exfat_init_fs_context(struct fs_context *fc)if (!sbi)return -ENOMEM ; -mutex_init (&sbi->s_lock); +init_rwsem (&sbi->s_lock);mutex_init (&sbi->bitmap_lock);ratelimit_state_init (&sbi->ratelimit,DEFAULT_RATELIMIT_INTERVAL ,DEFAULT_RATELIMIT_BURST );
Signature · 1 line
2.53.0
RFC PATCH 6/6 exfat: take s_lock in read mode for iomap mapping paths
In reply to Chi ZhilingFrom: Chi Zhiling <[email protected]>
The iomap mapping paths mainly access the FAT chain of the file itself. Concurrent mappings of distinct inodes can therefore proceed in parallel by taking s_lock in read mode instead of write mode. The superblock-wide state shared between files is already protected by the preceding patches: the allocation bitmap, used_clusters and clu_srch_ptr by bitmap_lock, and the volume dirty flag / boot sector by atomic bit ops with a single writer on the 0 -> 1 transition.
Concurrent access to the same inode stays serialized by the exclusive inode_lock held in exfat_file_write_iter().
Writeback takes the read lock even though it does not hold inode_lock. Folios under writeback are marked writeback, and truncate first flushes and truncates the page cache, so truncate cannot run concurrently with writeback. Writeback therefore only maps clusters that are still committed or owned by the inode and cannot race with cluster freeing.
The lock ordering remains inode_lock -> s_lock -> bitmap_lock, so this change does not introduce any new deadlock scenarios.
--- fs/exfat/iomap.c | 4 ++-- fs/exfat/super.c | 4 ++++ 2 files changed, 6 insertions(+), 2 deletions(-)
fs/exfat/iomap.c +2 -2
@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length num_clusters =exfat_bytes_to_cluster_round_up (sbi, offset + length) -exfat_bytes_to_cluster (sbi, offset); -down_write (&sbi->s_lock); +down_read (&sbi->s_lock); iomap->bdev = inode->i_sb->s_bdev; iomap->offset = offset;
@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length iomap->flags |=IOMAP_F_MERGED ; out: -up_write (&sbi->s_lock); +up_read (&sbi->s_lock);return err; }
fs/exfat/super.c +4 -0
@@ -94,6 +94,8 @@ int exfat_set_volume_dirty(struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb); +lockdep_assert_held (&sbi->s_lock); +if (test_and_set_bit (VOLUME_DIRTY_BIT , &sbi->vol_flags))return 0 ;
@@ -104,6 +106,8 @@ int exfat_clear_volume_dirty(struct super_block *sb) {struct exfat_sb_info *sbi =EXFAT_SB (sb); +lockdep_assert_held_write (&sbi->s_lock); +if (!test_and_clear_bit (VOLUME_DIRTY_BIT , &sbi->vol_flags))return 0 ;
Signature · 1 line
2.53.0
Namjae Jeon
maintainerfs/exfat/iomap.c static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length · line 71
Quoted 6 lines
fs/exfat/iomap.c +1
-1@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length num_clusters =exfat_bytes_to_cluster_round_up (sbi, offset + length) -exfat_bytes_to_cluster (sbi, offset); -down_write (&sbi->s_lock); +down_read (&sbi->s_lock);
If may_alloc is true, Should we use down_write() ?
Quoted 11 lines
fs/exfat/iomap.c +1
-1iomap->bdev = inode->i_sb->s_bdev; iomap->offset = offset;@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length iomap->flags |=IOMAP_F_MERGED ; out: -up_write (&sbi->s_lock); +up_read (&sbi->s_lock);return err; }
Quoted 7 lines · Namjae Jeon
fs/exfat/iomap.c static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length · line 71
Quoted 6 lines
fs/exfat/iomap.c +1
-1@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length num_clusters =exfat_bytes_to_cluster_round_up (sbi, offset + length) -exfat_bytes_to_cluster (sbi, offset); -down_write (&sbi->s_lock); +down_read (&sbi->s_lock);If
may_allocis true, Should we usedown_write()?
IMO, there is no need for down_write(); the block mapping of an inode is protected by the inode lock, not s_lock. see exfat_fallocate(), where we don't hold s_lock.
If may_alloc is true, we need to allocate clusters and append them to the file's FAT chain. The allocation is protected by bitmap_lock, while the FAT chain is protected by the inode lock, since we hold the inode write lock in exfat_file_write_iter(), exfat_fallocate() and exfat_truncate(), there is no race when modifying the FAT chain.
Besides, I'm trying to remove s_lock in the next version and am considering adding a per-inode lock to exfat_inode_info, since the inode lock has a relatively large scope.
Thanks,
Quoted 12 lines
Quoted 12 lines
fs/exfat/iomap.c +1
-1iomap->bdev = inode->i_sb->s_bdev; iomap->offset = offset;@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length iomap->flags |=IOMAP_F_MERGED ; out: -up_write (&sbi->s_lock); +up_read (&sbi->s_lock);return err; }
Namjae Jeon
maintainerQuoted 6 lines · Chi Zhiling
Quoted 4 lines · Namjae Jeon
Quoted 4 lines
fs/exfat/iomap.c +0
-0@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length num_clusters =exfat_bytes_to_cluster_round_up (sbi, offset + length) -exfat_bytes_to_cluster (sbi, offset);
Quoted 7 lines
Quoted 3 lines
fs/exfat/iomap.c static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length · line 71
Quoted 2 lines
fs/exfat/iomap.c +1
-1-down_write (&sbi->s_lock); +down_read (&sbi->s_lock);If
may_allocis true, Should we usedown_write()?IMO, there is no need for
down_write(); the block mapping of an inode is protected by the inode lock, nots_lock. seeexfat_fallocate(), where we don't holds_lock.
Okay.
Quoted 6 lines
If
may_allocis true, we need to allocate clusters and append them to the file's FAT chain. The allocation is protected bybitmap_lock, while the FAT chain is protected by the inode lock, since we hold the inode write lock inexfat_file_write_iter(),exfat_fallocate()andexfat_truncate(), there is no race when modifying the FAT chain.
Okay.
Quoted 4 lines
Besides, I'm trying to remove
s_lockin the next version and am considering adding a per-inode lock toexfat_inode_info, since the inode lock has a relatively large scope.
Okay, Sounds great!
Thanks!