Skip to contentSkip to conversation

block: avoid to serialize blkdev_fallocate

PATCH In review Ming Lei 4 participants 4 messages May 12, 2022 – May 31, 2022 Block layer

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

PATCH 1/1 block: avoid to serialize blkdev_fallocate

Commit f278eb3d8178 ("block: hold ->invalidate_lock in blkdev_fallocate") adds ->invalidate_lock in blkdev_fallocate() for preventing stale data from being read during fallocate().

However, the side effect is that blkdev_fallocate() becomes serialized since blkdev_fallocate() always blocks.

Add one atomic fallocate counter for allowing blkdev_fallocate() to be run concurrently so that discard/write_zero bios from different fallocate() can be submitted in parallel.

---
 block/fops.c              | 6 ++++--
 include/linux/blk_types.h | 2 ++
 2 files changed, 6 insertions(+), 2 deletions(-)
block/fops.c +4 -2
index 9f2ecec406b0..368866b15e68 100644
@@ -651,7 +651,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,
 	if ((start | len) & (bdev_logical_block_size(bdev) - 1))
 		return -EINVAL;
 
-	filemap_invalidate_lock(inode->i_mapping);
+	if (atomic_inc_return(&bdev->bd_fallocate_count) == 1)
+		filemap_invalidate_lock(inode->i_mapping);
 
 	/* Invalidate the page cache, including dirty pages. */
 	error = truncate_bdev_range(bdev, file->f_mode, start, end);
@@ -679,7 +680,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,
 	}
 
  fail:
-	filemap_invalidate_unlock(inode->i_mapping);
+	if (!atomic_dec_return(&bdev->bd_fallocate_count))
+		filemap_invalidate_unlock(inode->i_mapping);
 	return error;
 }
 
include/linux/blk_types.h +2 -0
index 1973ef9bd40f..9ccd841ea8ed 100644
@@ -58,6 +58,8 @@ struct block_device {
 	struct gendisk *	bd_disk;
 	struct request_queue *	bd_queue;
 
+	atomic_t		bd_fallocate_count;
+
 	/* The counter of freeze processes */
 	int			bd_fsfreeze_count;
 	/* Mutex for freeze */
Signature · 1 line
2.31.1
#1 lore raw
Quoted 10 lines · Ming Lei

Commit f278eb3d8178 ("block: hold ->invalidate_lock in blkdev_fallocate") adds ->invalidate_lock in blkdev_fallocate() for preventing stale data from being read during fallocate().

However, the side effect is that blkdev_fallocate() becomes serialized since blkdev_fallocate() always blocks.

Add one atomic fallocate counter for allowing blkdev_fallocate() to be run concurrently so that discard/write_zero bios from different fallocate() can be submitted in parallel.

Please consider adding a "Fixes:" tag since otherwise this patch won't be integrated automatically into any of the stable kernel trees.

Thanks,

Bart.

#2 lore raw
In reply to Ming Lei
Quoted 7 lines · Ming Lei

Commit f278eb3d8178 ("block: hold ->invalidate_lock in blkdev_fallocate") adds ->invalidate_lock in blkdev_fallocate() for preventing stale data from being read during fallocate().

However, the side effect is that blkdev_fallocate() becomes serialized since blkdev_fallocate() always blocks.

Quoted 6 lines

Add one atomic fallocate counter for allowing blkdev_fallocate() to be run concurrently so that discard/write_zero bios from different fallocate() can be submitted in parallel.

Couple of points:
* What about blk_ioctl_zeroout() or blk_ioctl_discard()?
* This way processes calling discard / zeroout can easily starve process wanting to do read which does not seem ideal. Somewhat related is that I know that Christoph wanted to modify how we use filemap_invalidate_lock() so that huge zeroouts performed by actually writing zeros will not block readers for ages and this seems to be going in somewhat opposite direction favoring writers even more.
* I suspect this is going to upset lockdep (and lock owner tracking for the purposes of spinning) pretty badly because the lock owner may be returning to userspace without unlocking the lock.

Honza

Quoted 45 lines
---
 block/fops.c              | 6 ++++--
 include/linux/blk_types.h | 2 ++
 2 files changed, 6 insertions(+), 2 deletions(-)
block/fops.c +4 -2
index 9f2ecec406b0..368866b15e68 100644
@@ -651,7 +651,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,
 	if ((start | len) & (bdev_logical_block_size(bdev) - 1))
 		return -EINVAL;
 
-	filemap_invalidate_lock(inode->i_mapping);
+	if (atomic_inc_return(&bdev->bd_fallocate_count) == 1)
+		filemap_invalidate_lock(inode->i_mapping);
 
 	/* Invalidate the page cache, including dirty pages. */
 	error = truncate_bdev_range(bdev, file->f_mode, start, end);
@@ -679,7 +680,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,
 	}
 
  fail:
-	filemap_invalidate_unlock(inode->i_mapping);
+	if (!atomic_dec_return(&bdev->bd_fallocate_count))
+		filemap_invalidate_unlock(inode->i_mapping);
 	return error;
 }
 
include/linux/blk_types.h +2 -0
index 1973ef9bd40f..9ccd841ea8ed 100644
@@ -58,6 +58,8 @@ struct block_device {
 	struct gendisk *	bd_disk;
 	struct request_queue *	bd_queue;
 
+	atomic_t		bd_fallocate_count;
+
 	/* The counter of freeze processes */
 	int			bd_fsfreeze_count;
 	/* Mutex for freeze */
Signature · 1 line
2.31.1
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#3 lore raw
Quoted 9 lines · Jan Kara

* This way processes calling discard / zeroout can easily starve process wanting to do read which does not seem ideal. Somewhat related is that I know that Christoph wanted to modify how we use filemap_invalidate_lock() so that huge zeroouts performed by actually writing zeros will not block readers for ages and this seems to be going in somewhat opposite direction favoring writers even more.
* I suspect this is going to upset lockdep (and lock owner tracking for the purposes of spinning) pretty badly because the lock owner may be returning to userspace without unlocking the lock.

Yes to all of the above. I think we need to just pick a smaller defauly zeroout size (and make that tunable) to avoid holding the lock over really long operations. Or turn the invalidate lock into a range lock..