Skip to contentSkip to conversation

exfat: take s_lock in read mode for iomap mapping paths

RFC PATCH In review Chi Zhiling 2 participants 12 messages Aug 21 – Aug 27 exFAT file system

lore
Share & tools
lore
#0 lore raw

cover exfat: take s_lock in read mode for iomap mapping paths

From: Chi Zhiling <[email protected]>

The iomap mapping paths currently serialize all inode mappings on the sbi->s_lock mutex. This series converts it to an rw_semaphore and takes it in read mode on the mapping paths so that mappings of distinct inodes proceed in parallel. The preceding patches make the paths safe to run
concurrently: bitmap_lock covers the allocation state, atomic bit ops cover the volume dirty flag, and the FAT2 mirror copy is serialized against writeback.

Same-inode access stays serialized by the exclusive inode_lock; writeback does not hold it but is safe because truncate flushes and truncates the page cache first and cannot run concurrently with writeback. The lock ordering (inode_lock -> s_lock -> bitmap_lock) is unchanged, so no new deadlock scenarios are introduced.

A per-inode read-write lock was also considered, but its implementation turned out to be considerably more complex, so this series relaxes the existing s_lock instead. Any suggestions on that approach are welcome.

The series passes the xfstests exfat suite. In Unixbench, the fstime-w score improved from 3977196.8 to 4746349.6 (~+19%). Any comments and suggestions are welcome.

Chi Zhiling (6):
exfat: remove dead hint_bmap updates in I/O and truncate paths
exfat: take bitmap_lock at the start of exfat_alloc_cluster()
exfat: use atomic bit ops for volume dirty flag
exfat: lock FAT2 buffer while copying mirrored FAT entries
exfat: convert s_lock mutex to rw_semaphore using write lock
exfat: take s_lock in read mode for iomap mapping paths

 fs/exfat/dir.c       | 14 +++++++-------
 fs/exfat/exfat_fs.h  |  5 ++---
 fs/exfat/exfat_raw.h |  4 ++--
 fs/exfat/fatent.c    | 20 +++++++++++++-------
 fs/exfat/file.c      |  8 ++------
 fs/exfat/inode.c     | 15 +++++----------
 fs/exfat/iomap.c     |  4 ++--
 fs/exfat/namei.c     | 26 +++++++++++++-------------
 fs/exfat/super.c     | 42 +++++++++++++++++++++---------------------
 9 files changed, 67 insertions(+), 71 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

RFC PATCH 1/6 exfat: remove dead hint_bmap updates in I/O and truncate paths

From: Chi Zhiling <[email protected]>

hint_bmap is only consumed by exfat_readdir() during directory iteration. exfat_map_cluster() runs on the file I/O path (via iomap) and never reaches directory iteration, and __exfat_truncate() only resets the hint when the inode is already being evicted, so drop them along with the now-unused local_clu_offset.

---
 fs/exfat/file.c  | 4 ----
 fs/exfat/inode.c | 5 -----
 2 files changed, 9 deletions(-)
fs/exfat/file.c +0 -4
index a2a9ee1a2004..29c5db12288e 100644
@@ -278,10 +278,6 @@ int __exfat_truncate(struct inode *inode)
 	/* clear exfat cache */
 	exfat_cache_inval_inode(inode);
 
-	/* hint information */
-	ei->hint_bmap.off = EXFAT_EOF_CLUSTER;
-	ei->hint_bmap.clu = EXFAT_EOF_CLUSTER;
-
 	/* hint_stat will be used if this is directory. */
 	ei->hint_stat.eidx = 0;
 	ei->hint_stat.clu = ei->start_clu;
fs/exfat/inode.c +0 -5
index ccd13630187e..d041f893b1c3 100644
@@ -144,7 +144,6 @@ int exfat_map_cluster(struct inode *inode, unsigned int clu_offset,
 	struct super_block *sb = inode->i_sb;
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 	struct exfat_inode_info *ei = EXFAT_I(inode);
-	unsigned int local_clu_offset = clu_offset;
 	unsigned int num_to_be_allocated = 0, num_clusters;
 
 	num_clusters = exfat_bytes_to_cluster(sbi, exfat_ondisk_size(inode));
@@ -234,10 +233,6 @@ int exfat_map_cluster(struct inode *inode, unsigned int clu_offset,
 			*balloc = true;
 	}
 
-	/* hint information */
-	ei->hint_bmap.off = local_clu_offset;
-	ei->hint_bmap.clu = *clu;
-
 	return 0;
 }
 
Signature · 1 line
2.53.0
#2 lore raw

RFC PATCH 2/6 exfat: take bitmap_lock at the start of exfat_alloc_cluster()

In reply to Chi Zhiling

From: Chi Zhiling <[email protected]>

Acquire sbi->bitmap_lock at the top of exfat_alloc_cluster() so the used_clusters/clu_srch_ptr validation and updates are covered by the lock, removing a window where the early checks raced with concurrent allocations.

---
 fs/exfat/fatent.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
fs/exfat/fatent.c +10 -7
index a8b11e2ce43f..2db959669563 100644
@@ -427,19 +427,22 @@ int exfat_alloc_cluster(struct inode *inode, unsigned int num_alloc,
 	struct super_block *sb = inode->i_sb;
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 
+	mutex_lock(&sbi->bitmap_lock);
+
 	total_cnt = EXFAT_DATA_CLUSTER_COUNT(sbi);
 
 	if (unlikely(total_cnt < sbi->used_clusters)) {
 		exfat_fs_error_ratelimit(sb,
 			"%s: invalid used clusters(t:%u,u:%u)\n",
 			__func__, total_cnt, sbi->used_clusters);
-		return -EIO;
+		ret = -EIO;
+		goto unlock;
 	}
 
-	if (num_alloc > total_cnt - sbi->used_clusters)
-		return -ENOSPC;
-
-	mutex_lock(&sbi->bitmap_lock);
+	if (num_alloc > total_cnt - sbi->used_clusters) {
+		ret = -ENOSPC;
+		goto unlock;
+	}
 
 	hint_clu = p_chain->dir;
 	/* find new cluster */
@@ -516,8 +519,8 @@ int exfat_alloc_cluster(struct inode *inode, unsigned int num_alloc,
 done:
 			sbi->clu_srch_ptr = hint_clu;
 			sbi->used_clusters += p_chain->size;
-			mutex_unlock(&sbi->bitmap_lock);
-			return 0;
+			ret = 0;
+			goto unlock;
 		}
 
 		hint_clu = new_clu + 1;
Signature · 1 line
2.53.0
#3 lore raw

RFC PATCH 3/6 exfat: use atomic bit ops for volume dirty flag

In reply to Chi Zhiling

From: Chi Zhiling <[email protected]>

After converting s_lock to a reader-writer lock, multiple operations can execute concurrently under the read lock. As a result, multiple threads may concurrently update sbi->vol_flags, causing a race in the existing read-modify-write sequence.

Convert sbi->vol_flags to an unsigned long bitmap and use test_and_set_bit() / test_and_clear_bit() to atomically update the volume dirty bit. Only the thread performing the 0 -> 1 transition updates the boot sector, avoiding concurrent unsynchronized modifications of vol_flags and the boot-sector buffer.

Remove vol_flags_persistent, since the bitmap can directly retain the MEDIA_FAILURE state while preserving the complete 16-bit on-disk volume flags field when writing it back.

---
 fs/exfat/exfat_fs.h  |  3 +--
 fs/exfat/exfat_raw.h |  4 ++--
 fs/exfat/super.c     | 28 ++++++++++++----------------
 3 files changed, 15 insertions(+), 20 deletions(-)
fs/exfat/exfat_fs.h +1 -2
index a9131fe03302..f1505c013248 100644
@@ -232,8 +232,7 @@ struct exfat_sb_info {
 	unsigned int num_FAT_sectors; /* num of FAT sectors */
 	unsigned int root_dir; /* root dir cluster */
 	unsigned int dentries_per_clu; /* num of dentries per cluster */
-	unsigned int vol_flags; /* volume flags */
-	unsigned int vol_flags_persistent; /* volume flags to retain */
+	unsigned long vol_flags; /* volume flags (bitmap) */
 	struct buffer_head *boot_bh; /* buffer_head of BOOT sector */
 
 	unsigned int map_clu; /* allocation bitmap start cluster */
fs/exfat/exfat_raw.h +2 -2
index ec70cd35bba0..222fee5c2adf 100644
@@ -14,8 +14,8 @@
 
 #define EXFAT_MAX_FILE_LEN	255
 
-#define VOLUME_DIRTY		0x0002
-#define MEDIA_FAILURE		0x0004
+#define VOLUME_DIRTY_BIT	1
+#define MEDIA_FAILURE_BIT	2
 
 #define EXFAT_EOF_CLUSTER	0xFFFFFFFFu
 #define EXFAT_BAD_CLUSTER	0xFFFFFFF7u
fs/exfat/super.c +12 -16
index a9ea36ba2693..491273d8eeb6 100644
@@ -69,27 +69,18 @@ static int exfat_statfs(struct dentry *dentry, struct kstatfs *buf)
 	return 0;
 }
 
-static int exfat_set_vol_flags(struct super_block *sb, unsigned short new_flags)
+static int exfat_sync_vol_flags(struct super_block *sb)
 {
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 	struct boot_sector *p_boot = (struct boot_sector *)sbi->boot_bh->b_data;
 
-	/* retain persistent-flags */
-	new_flags |= sbi->vol_flags_persistent;
-
-	/* flags are not changed */
-	if (sbi->vol_flags == new_flags)
-		return 0;
-
-	sbi->vol_flags = new_flags;
-
 	/* skip updating volume dirty flag,
 	 * if this volume has been mounted with read-only
 	 */
 	if (sb_rdonly(sb))
 		return 0;
 
-	p_boot->vol_flags = cpu_to_le16(new_flags);
+	p_boot->vol_flags = cpu_to_le16((unsigned short)READ_ONCE(sbi->vol_flags));
 
 	set_buffer_uptodate(sbi->boot_bh);
 	mark_buffer_dirty(sbi->boot_bh);
@@ -103,14 +94,20 @@ int exfat_set_volume_dirty(struct super_block *sb)
 {
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 
-	return exfat_set_vol_flags(sb, sbi->vol_flags | VOLUME_DIRTY);
+	if (test_and_set_bit(VOLUME_DIRTY_BIT, &sbi->vol_flags))
+		return 0;
+
+	return exfat_sync_vol_flags(sb);
 }
 
 int exfat_clear_volume_dirty(struct super_block *sb)
 {
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 
-	return exfat_set_vol_flags(sb, sbi->vol_flags & ~VOLUME_DIRTY);
+	if (!test_and_clear_bit(VOLUME_DIRTY_BIT, &sbi->vol_flags))
+		return 0;
+
+	return exfat_sync_vol_flags(sb);
 }
 
 static int exfat_show_options(struct seq_file *m, struct dentry *root)
@@ -509,7 +506,6 @@ static int exfat_read_boot_sector(struct super_block *sb)
 		(sbi->cluster_size_bits - DENTRY_SIZE_BITS);
 
 	sbi->vol_flags = le16_to_cpu(p_boot->vol_flags);
-	sbi->vol_flags_persistent = sbi->vol_flags & (VOLUME_DIRTY | MEDIA_FAILURE);
 	sbi->clu_srch_ptr = EXFAT_FIRST_CLUSTER;
 
 	/* check consistencies */
@@ -526,9 +522,9 @@ static int exfat_read_boot_sector(struct super_block *sb)
 		return -EINVAL;
 	}
 
-	if (sbi->vol_flags & VOLUME_DIRTY)
+	if (test_bit(VOLUME_DIRTY_BIT, &sbi->vol_flags))
 		exfat_warn(sb, "Volume was not properly unmounted. Some data may be corrupt. Please run fsck.");
-	if (sbi->vol_flags & MEDIA_FAILURE)
+	if (test_bit(MEDIA_FAILURE_BIT, &sbi->vol_flags))
 		exfat_warn(sb, "Medium has reported failures. Some data may be lost.");
 
 	/*
Signature · 1 line
2.53.0
maintainer
#4 lore raw

Re: [RFC PATCH v1 3/6] exfat: use atomic bit ops for volume dirty flag

fs/exfat/super.c static int exfat_statfs(struct dentry *dentry, struct kstatfs *buf) · line 78

Quoted 8 lines
fs/exfat/super.c +1 -3
-static int exfat_set_vol_flags(struct super_block *sb, unsigned short new_flags)
+static int exfat_sync_vol_flags(struct super_block *sb)
 {
        struct exfat_sb_info *sbi = EXFAT_SB(sb);
        struct boot_sector *p_boot = (struct boot_sector *)sbi->boot_bh->b_data;
 
-       /* retain persistent-flags */
-       new_flags |= sbi->vol_flags_persistent;

Please don't remove ->vol_flags_persistent. It prevents pre-existing volume dirty flags from being lost, but removing it clears a dirty flag set before mount during normal unmount.

#5 lore raw

Re: [RFC PATCH v1 3/6] exfat: use atomic bit ops for volume dirty flag

Hi, Namjae

You are right. This should not be removed. I will restore it in the next version.

Thanks,

Quoted 11 lines · Namjae Jeon

fs/exfat/super.c static int exfat_statfs(struct dentry *dentry, struct kstatfs *buf) · line 78

Quoted 8 lines
fs/exfat/super.c +1 -3
-static int exfat_set_vol_flags(struct super_block *sb, unsigned short new_flags)
+static int exfat_sync_vol_flags(struct super_block *sb)
  {
         struct exfat_sb_info *sbi = EXFAT_SB(sb);
         struct boot_sector *p_boot = (struct boot_sector *)sbi->boot_bh->b_data;
 
-       /* retain persistent-flags */
-       new_flags |= sbi->vol_flags_persistent;

Please don't remove ->vol_flags_persistent. It prevents pre-existing volume dirty flags from being lost, but removing it clears a dirty flag set before mount during normal unmount.

#6 lore raw

RFC PATCH 4/6 exfat: lock FAT2 buffer while copying mirrored FAT entries

In reply to Chi Zhiling

From: Chi Zhiling <[email protected]>

exfat_mirror_bh() updates the FAT2 buffer without serializing the copy against writeback. Once s_lock is converted to a read-write lock, multiple read-side operations may run concurrently with writeback. This can allow writeback to observe a partially updated FAT2 block.

Lock the destination buffer while copying the FAT entry data to serialize the update with writeback and prevent torn FAT2 writes.

---
 fs/exfat/fatent.c | 3 +++
 1 file changed, 3 insertions(+)
fs/exfat/fatent.c +3 -0
index 2db959669563..f6ad2825ad19 100644
@@ -24,7 +24,10 @@ static int exfat_mirror_bh(struct super_block *sb, struct buffer_head *bh)
 		c_bh = sb_getblk(sb, sec2);
 		if (!c_bh)
 			return -ENOMEM;
+		/* Serialize the copy with writeback to avoid a torn FAT2 write */
+		lock_buffer(c_bh);
 		memcpy(c_bh->b_data, bh->b_data, sb->s_blocksize);
+		unlock_buffer(c_bh);
 		err = exfat_update_bh(c_bh, sb->s_flags & SB_SYNCHRONOUS);
 		brelse(c_bh);
 	}
Signature · 1 line
2.53.0
#7 lore raw

RFC PATCH 5/6 exfat: convert s_lock mutex to rw_semaphore using write lock

In reply to Chi Zhiling

From: Chi Zhiling <[email protected]>

Convert sbi->s_lock from a mutex to an rw_semaphore and acquire it in write mode at every existing use site. This is a mechanical conversion with no behavioral change, preparing the lock for read-mode access in the iomap mapping paths so that concurrent mappings of distinct inodes can proceed in parallel.

Lock ordering stays consistent (inode_lock -> s_lock -> bitmap_lock), so no deadlock is introduced.

---
 fs/exfat/dir.c      | 14 +++++++-------
 fs/exfat/exfat_fs.h |  2 +-
 fs/exfat/file.c     |  4 ++--
 fs/exfat/inode.c    | 10 +++++-----
 fs/exfat/iomap.c    |  4 ++--
 fs/exfat/namei.c    | 26 +++++++++++++-------------
 fs/exfat/super.c    | 10 +++++-----
 7 files changed, 35 insertions(+), 35 deletions(-)
fs/exfat/dir.c +7 -7
index fe73b1380c5d..ea454574260e 100644
@@ -223,7 +223,7 @@ static int exfat_iterate(struct file *file, struct dir_context *ctx)
 	if (err)
 		goto out;
 get_new:
-	mutex_lock(&EXFAT_SB(sb)->s_lock);
+	down_write(&EXFAT_SB(sb)->s_lock);
 
 	if (ei->flags == ALLOC_NO_FAT_CHAIN && cpos >= i_size_read(inode))
 		goto end_of_dir;
@@ -255,7 +255,7 @@ static int exfat_iterate(struct file *file, struct dir_context *ctx)
 		inum = iunique(sb, EXFAT_ROOT_INO);
 	}
 
-	mutex_unlock(&EXFAT_SB(sb)->s_lock);
+	up_write(&EXFAT_SB(sb)->s_lock);
 	if (!dir_emit(ctx, nb->lfn, strlen(nb->lfn), inum,
 			(de.attr & EXFAT_ATTR_SUBDIR) ? DT_DIR : DT_REG))
 		goto out;
@@ -266,7 +266,7 @@ static int exfat_iterate(struct file *file, struct dir_context *ctx)
 	if (!cpos && fake_offset)
 		cpos = ITER_POS_FILLED_DOTS;
 	ctx->pos = cpos;
-	mutex_unlock(&EXFAT_SB(sb)->s_lock);
+	up_write(&EXFAT_SB(sb)->s_lock);
 out:
 	/*
 	 * To improve performance, free namebuf after unlock sb_lock.
@@ -1294,7 +1294,7 @@ int exfat_read_volume_label(struct super_block *sb, struct exfat_uni_name *label
 	struct exfat_entry_set_cache es;
 	struct exfat_dentry *ep;
 
-	mutex_lock(&sbi->s_lock);
+	down_write(&sbi->s_lock);
 
 	memset(label_out, 0, sizeof(*label_out));
 	ret = exfat_get_volume_label_dentry(sb, &es);
@@ -1322,7 +1322,7 @@ int exfat_read_volume_label(struct super_block *sb, struct exfat_uni_name *label
 
 	exfat_put_dentry_set(&es, false);
 unlock:
-	mutex_unlock(&sbi->s_lock);
+	up_write(&sbi->s_lock);
 	return ret;
 }
 
@@ -1339,7 +1339,7 @@ int exfat_write_volume_label(struct super_block *sb,
 	if (label->name_len > EXFAT_VOLUME_LABEL_LEN)
 		return -EINVAL;
 
-	mutex_lock(&sbi->s_lock);
+	down_write(&sbi->s_lock);
 
 	ret = exfat_get_volume_label_dentry(sb, &es);
 	if (ret == -ENOENT) {
@@ -1376,6 +1376,6 @@ int exfat_write_volume_label(struct super_block *sb,
 	ret = exfat_put_dentry_set(&es, IS_DIRSYNC(root_inode));
 
 unlock:
-	mutex_unlock(&sbi->s_lock);
+	up_write(&sbi->s_lock);
 	return ret;
 }
fs/exfat/exfat_fs.h +1 -1
index f1505c013248..16ce508e0946 100644
@@ -246,7 +246,7 @@ struct exfat_sb_info {
 
 	unsigned long s_exfat_flags; /* Exfat superblock flags */
 
-	struct mutex s_lock; /* superblock lock */
+	struct rw_semaphore s_lock; /* superblock lock */
 	struct mutex bitmap_lock; /* bitmap lock */
 	struct exfat_mount_options options;
 	struct nls_table *nls_io; /* Charset used for input and display */
fs/exfat/file.c +2 -2
index 29c5db12288e..8abef3d2294a 100644
@@ -297,7 +297,7 @@ static void exfat_truncate(struct inode *inode)
 	struct exfat_inode_info *ei = EXFAT_I(inode);
 	int err;
 
-	mutex_lock(&sbi->s_lock);
+	down_write(&sbi->s_lock);
 	if (ei->start_clu == 0) {
 		/*
 		 * Empty start_clu != ~0 (not allocated)
@@ -312,7 +312,7 @@ static void exfat_truncate(struct inode *inode)
 
 	inode->i_blocks = round_up(i_size_read(inode), sbi->cluster_size) >> 9;
 write_size:
-	mutex_unlock(&sbi->s_lock);
+	up_write(&sbi->s_lock);
 }
 
 int exfat_getattr(struct mnt_idmap *idmap, const struct path *path,
fs/exfat/inode.c +5 -5
index d041f893b1c3..a89826e08e1c 100644
@@ -116,16 +116,16 @@ int exfat_write_inode(struct inode *inode, struct writeback_control *wbc)
 	if (unlikely(exfat_forced_shutdown(inode->i_sb)))
 		return -EIO;
 
-	mutex_lock(&EXFAT_SB(inode->i_sb)->s_lock);
+	down_write(&EXFAT_SB(inode->i_sb)->s_lock);
 	ret = __exfat_write_inode(inode, wbc->sync_mode == WB_SYNC_ALL);
-	mutex_unlock(&EXFAT_SB(inode->i_sb)->s_lock);
+	up_write(&EXFAT_SB(inode->i_sb)->s_lock);
 
 	return ret;
 }
 
 void exfat_sync_inode(struct inode *inode)
 {
-	lockdep_assert_held(&EXFAT_SB(inode->i_sb)->s_lock);
+	lockdep_assert_held_write(&EXFAT_SB(inode->i_sb)->s_lock);
 	__exfat_write_inode(inode, 1);
 }
 
@@ -443,9 +443,9 @@ void exfat_evict_inode(struct inode *inode)
 
 	if (!inode->i_nlink) {
 		i_size_write(inode, 0);
-		mutex_lock(&EXFAT_SB(inode->i_sb)->s_lock);
+		down_write(&EXFAT_SB(inode->i_sb)->s_lock);
 		__exfat_truncate(inode);
-		mutex_unlock(&EXFAT_SB(inode->i_sb)->s_lock);
+		up_write(&EXFAT_SB(inode->i_sb)->s_lock);
 	}
 
 	clear_inode(inode);
fs/exfat/iomap.c +2 -2
index 0c805bf6676a..bc8bdfa9bb80 100644
@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
 	num_clusters = exfat_bytes_to_cluster_round_up(sbi,
 			offset + length) - exfat_bytes_to_cluster(sbi, offset);
 
-	mutex_lock(&sbi->s_lock);
+	down_write(&sbi->s_lock);
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->offset = offset;
 
@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
 
 	iomap->flags |= IOMAP_F_MERGED;
 out:
-	mutex_unlock(&sbi->s_lock);
+	up_write(&sbi->s_lock);
 	return err;
 }
 
fs/exfat/namei.c +13 -13
index f26f987a34cf..0c28040e97d6 100644
@@ -565,7 +565,7 @@ static int exfat_create(struct mnt_idmap *idmap, struct inode *dir,
 	if (unlikely(exfat_forced_shutdown(sb)))
 		return -EIO;
 
-	mutex_lock(&EXFAT_SB(sb)->s_lock);
+	down_write(&EXFAT_SB(sb)->s_lock);
 	exfat_set_volume_dirty(sb);
 	err = exfat_add_entry(dir, dentry->d_name.name, TYPE_FILE, &info);
 	if (err)
@@ -592,7 +592,7 @@ static int exfat_create(struct mnt_idmap *idmap, struct inode *dir,
 
 	d_instantiate(dentry, inode);
 unlock:
-	mutex_unlock(&EXFAT_SB(sb)->s_lock);
+	up_write(&EXFAT_SB(sb)->s_lock);
 	return err;
 }
 
@@ -732,7 +732,7 @@ static struct dentry *exfat_lookup(struct inode *dir, struct dentry *dentry,
 	int err;
 	loff_t i_pos;
 
-	mutex_lock(&EXFAT_SB(sb)->s_lock);
+	down_write(&EXFAT_SB(sb)->s_lock);
 	err = exfat_find(dir, &dentry->d_name, &info);
 	if (err) {
 		if (unlikely(err != -ENOENT))
@@ -761,12 +761,12 @@ static struct dentry *exfat_lookup(struct inode *dir, struct dentry *dentry,
 		 */
 		d_move(alias, dentry);
 		iput(inode);
-		mutex_unlock(&EXFAT_SB(sb)->s_lock);
+		up_write(&EXFAT_SB(sb)->s_lock);
 		return alias;
 	}
 	dput(alias);
 out:
-	mutex_unlock(&EXFAT_SB(sb)->s_lock);
+	up_write(&EXFAT_SB(sb)->s_lock);
 	if (!inode)
 		exfat_d_version_set(dentry, inode_query_iversion(dir));
 
@@ -785,7 +785,7 @@ static int exfat_unlink(struct inode *dir, struct dentry *dentry)
 	if (unlikely(exfat_forced_shutdown(sb)))
 		return -EIO;
 
-	mutex_lock(&EXFAT_SB(sb)->s_lock);
+	down_write(&EXFAT_SB(sb)->s_lock);
 	if (ei->dir.dir == DIR_DELETED) {
 		exfat_err(sb, "abnormal access to deleted dentry");
 		err = -ENOENT;
@@ -821,7 +821,7 @@ static int exfat_unlink(struct inode *dir, struct dentry *dentry)
 	exfat_unhash_inode(inode);
 	exfat_d_version_set(dentry, inode_query_iversion(dir));
 unlock:
-	mutex_unlock(&EXFAT_SB(sb)->s_lock);
+	up_write(&EXFAT_SB(sb)->s_lock);
 	return err;
 }
 
@@ -838,7 +838,7 @@ static struct dentry *exfat_mkdir(struct mnt_idmap *idmap, struct inode *dir,
 	if (unlikely(exfat_forced_shutdown(sb)))
 		return ERR_PTR(-EIO);
 
-	mutex_lock(&EXFAT_SB(sb)->s_lock);
+	down_write(&EXFAT_SB(sb)->s_lock);
 	exfat_set_volume_dirty(sb);
 	err = exfat_add_entry(dir, dentry->d_name.name, TYPE_DIR, &info);
 	if (err)
@@ -867,7 +867,7 @@ static struct dentry *exfat_mkdir(struct mnt_idmap *idmap, struct inode *dir,
 	d_instantiate(dentry, inode);
 
 unlock:
-	mutex_unlock(&EXFAT_SB(sb)->s_lock);
+	up_write(&EXFAT_SB(sb)->s_lock);
 	return err ? ERR_PTR(err) : NULL;
 }
 
@@ -929,7 +929,7 @@ static int exfat_rmdir(struct inode *dir, struct dentry *dentry)
 	if (unlikely(exfat_forced_shutdown(sb)))
 		return -EIO;
 
-	mutex_lock(&EXFAT_SB(inode->i_sb)->s_lock);
+	down_write(&EXFAT_SB(inode->i_sb)->s_lock);
 
 	if (ei->dir.dir == DIR_DELETED) {
 		exfat_err(sb, "abnormal access to deleted dentry");
@@ -979,7 +979,7 @@ static int exfat_rmdir(struct inode *dir, struct dentry *dentry)
 	exfat_unhash_inode(inode);
 	exfat_d_version_set(dentry, inode_query_iversion(dir));
 unlock:
-	mutex_unlock(&EXFAT_SB(inode->i_sb)->s_lock);
+	up_write(&EXFAT_SB(inode->i_sb)->s_lock);
 	return err;
 }
 
@@ -1282,7 +1282,7 @@ static int exfat_rename(struct mnt_idmap *idmap,
 	if (flags & ~RENAME_NOREPLACE)
 		return -EINVAL;
 
-	mutex_lock(&EXFAT_SB(sb)->s_lock);
+	down_write(&EXFAT_SB(sb)->s_lock);
 	old_inode = old_dentry->d_inode;
 	new_inode = new_dentry->d_inode;
 
@@ -1334,7 +1334,7 @@ static int exfat_rename(struct mnt_idmap *idmap,
 	}
 
 unlock:
-	mutex_unlock(&EXFAT_SB(sb)->s_lock);
+	up_write(&EXFAT_SB(sb)->s_lock);
 	return err;
 }
 
fs/exfat/super.c +5 -5
index 491273d8eeb6..72a35f4079b4 100644
@@ -45,11 +45,11 @@ static void exfat_put_super(struct super_block *sb)
 {
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 
-	mutex_lock(&sbi->s_lock);
+	down_write(&sbi->s_lock);
 	exfat_clear_volume_dirty(sb);
 	exfat_free_bitmap(sbi);
 	brelse(sbi->boot_bh);
-	mutex_unlock(&sbi->s_lock);
+	up_write(&sbi->s_lock);
 }
 
 static int exfat_statfs(struct dentry *dentry, struct kstatfs *buf)
@@ -771,9 +771,9 @@ static int exfat_reconfigure(struct fs_context *fc)
 	fc->sb_flags |= SB_NODIRATIME;
 
 	sync_filesystem(sb);
-	mutex_lock(&sbi->s_lock);
+	down_write(&sbi->s_lock);
 	exfat_clear_volume_dirty(sb);
-	mutex_unlock(&sbi->s_lock);
+	up_write(&sbi->s_lock);
 
 	if (new_opts->allow_utime == (unsigned short)-1)
 		new_opts->allow_utime = ~new_opts->fs_dmask & 0022;
@@ -820,7 +820,7 @@ static int exfat_init_fs_context(struct fs_context *fc)
 	if (!sbi)
 		return -ENOMEM;
 
-	mutex_init(&sbi->s_lock);
+	init_rwsem(&sbi->s_lock);
 	mutex_init(&sbi->bitmap_lock);
 	ratelimit_state_init(&sbi->ratelimit, DEFAULT_RATELIMIT_INTERVAL,
 			DEFAULT_RATELIMIT_BURST);
Signature · 1 line
2.53.0
#8 lore raw

RFC PATCH 6/6 exfat: take s_lock in read mode for iomap mapping paths

In reply to Chi Zhiling

From: Chi Zhiling <[email protected]>

The iomap mapping paths mainly access the FAT chain of the file itself. Concurrent mappings of distinct inodes can therefore proceed in parallel by taking s_lock in read mode instead of write mode. The superblock-wide state shared between files is already protected by the preceding patches: the allocation bitmap, used_clusters and clu_srch_ptr by bitmap_lock, and the volume dirty flag / boot sector by atomic bit ops with a single writer on the 0 -> 1 transition.

Concurrent access to the same inode stays serialized by the exclusive inode_lock held in exfat_file_write_iter().

Writeback takes the read lock even though it does not hold inode_lock. Folios under writeback are marked writeback, and truncate first flushes and truncates the page cache, so truncate cannot run concurrently with writeback. Writeback therefore only maps clusters that are still committed or owned by the inode and cannot race with cluster freeing.

The lock ordering remains inode_lock -> s_lock -> bitmap_lock, so this change does not introduce any new deadlock scenarios.

---
 fs/exfat/iomap.c | 4 ++--
 fs/exfat/super.c | 4 ++++
 2 files changed, 6 insertions(+), 2 deletions(-)
fs/exfat/iomap.c +2 -2
index bc8bdfa9bb80..147e9da01a47 100644
@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
 	num_clusters = exfat_bytes_to_cluster_round_up(sbi,
 			offset + length) - exfat_bytes_to_cluster(sbi, offset);
 
-	down_write(&sbi->s_lock);
+	down_read(&sbi->s_lock);
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->offset = offset;
 
@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
 
 	iomap->flags |= IOMAP_F_MERGED;
 out:
-	up_write(&sbi->s_lock);
+	up_read(&sbi->s_lock);
 	return err;
 }
 
fs/exfat/super.c +4 -0
index 72a35f4079b4..64ec4d2d1bf6 100644
@@ -94,6 +94,8 @@ int exfat_set_volume_dirty(struct super_block *sb)
 {
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 
+	lockdep_assert_held(&sbi->s_lock);
+
 	if (test_and_set_bit(VOLUME_DIRTY_BIT, &sbi->vol_flags))
 		return 0;
 
@@ -104,6 +106,8 @@ int exfat_clear_volume_dirty(struct super_block *sb)
 {
 	struct exfat_sb_info *sbi = EXFAT_SB(sb);
 
+	lockdep_assert_held_write(&sbi->s_lock);
+
 	if (!test_and_clear_bit(VOLUME_DIRTY_BIT, &sbi->vol_flags))
 		return 0;
 
Signature · 1 line
2.53.0
maintainer
#9 lore raw

fs/exfat/iomap.c static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length · line 71

Quoted 6 lines
fs/exfat/iomap.c +1 -1
@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
        num_clusters = exfat_bytes_to_cluster_round_up(sbi,
                        offset + length) - exfat_bytes_to_cluster(sbi, offset);
 
-       down_write(&sbi->s_lock);
+       down_read(&sbi->s_lock);

If may_alloc is true, Should we use down_write() ?

Quoted 11 lines
fs/exfat/iomap.c +1 -1
        iomap->bdev = inode->i_sb->s_bdev;
        iomap->offset = offset;
 
@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
 
        iomap->flags |= IOMAP_F_MERGED;
 out:
-       up_write(&sbi->s_lock);
+       up_read(&sbi->s_lock);
        return err;
 }
#10 lore raw
Quoted 7 lines · Namjae Jeon

fs/exfat/iomap.c static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length · line 71

Quoted 6 lines
fs/exfat/iomap.c +1 -1
@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
         num_clusters = exfat_bytes_to_cluster_round_up(sbi,
                         offset + length) - exfat_bytes_to_cluster(sbi, offset);
 
-       down_write(&sbi->s_lock);
+       down_read(&sbi->s_lock);

If may_alloc is true, Should we use down_write() ?

IMO, there is no need for down_write(); the block mapping of an inode is protected by the inode lock, not s_lock. see exfat_fallocate(), where we don't hold s_lock.

If may_alloc is true, we need to allocate clusters and append them to the file's FAT chain. The allocation is protected by bitmap_lock, while the FAT chain is protected by the inode lock, since we hold the inode write lock in exfat_file_write_iter(), exfat_fallocate() and exfat_truncate(), there is no race when modifying the FAT chain.

Besides, I'm trying to remove s_lock in the next version and am considering adding a per-inode lock to exfat_inode_info, since the inode lock has a relatively large scope.

Thanks,

Quoted 12 lines
Quoted 12 lines
fs/exfat/iomap.c +1 -1
         iomap->bdev = inode->i_sb->s_bdev;
         iomap->offset = offset;
 
@@ -135,7 +135,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
 
         iomap->flags |= IOMAP_F_MERGED;
  out:
-       up_write(&sbi->s_lock);
+       up_read(&sbi->s_lock);
         return err;
  }
maintainer
#11 lore raw
Quoted 6 lines · Chi Zhiling
Quoted 4 lines · Namjae Jeon
Quoted 4 lines
fs/exfat/iomap.c +0 -0
@@ -68,7 +68,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
         num_clusters = exfat_bytes_to_cluster_round_up(sbi,
                         offset + length) - exfat_bytes_to_cluster(sbi, offset);
 
Quoted 7 lines
Quoted 3 lines

fs/exfat/iomap.c static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length · line 71

Quoted 2 lines
fs/exfat/iomap.c +1 -1
-       down_write(&sbi->s_lock);
+       down_read(&sbi->s_lock);

If may_alloc is true, Should we use down_write() ?

IMO, there is no need for down_write(); the block mapping of an inode is protected by the inode lock, not s_lock. see exfat_fallocate(), where we don't hold s_lock.

Okay.

Quoted 6 lines

If may_alloc is true, we need to allocate clusters and append them to the file's FAT chain. The allocation is protected by bitmap_lock, while the FAT chain is protected by the inode lock, since we hold the inode write lock in exfat_file_write_iter(), exfat_fallocate() and exfat_truncate(), there is no race when modifying the FAT chain.

Okay.

Quoted 4 lines

Besides, I'm trying to remove s_lock in the next version and am considering adding a per-inode lock to exfat_inode_info, since the inode lock has a relatively large scope.

Okay, Sounds great!
Thanks!