block: avoid to serialize blkdev_fallocate
PATCH 1/1 block: avoid to serialize blkdev_fallocate
Commit f278eb3d8178 ("block: hold ->invalidate_lock in blkdev_fallocate") adds ->invalidate_lock in blkdev_fallocate() for preventing stale data from being read during fallocate().
However, the side effect is that blkdev_fallocate() becomes serialized since blkdev_fallocate() always blocks.
Add one atomic fallocate counter for allowing blkdev_fallocate() to be run concurrently so that discard/write_zero bios from different fallocate() can be submitted in parallel.
--- block/fops.c | 6 ++++-- include/linux/blk_types.h | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-)
block/fops.c +4 -2
@@ -651,7 +651,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,if ((start | len) & (bdev_logical_block_size (bdev) -1 ))return -EINVAL ; -filemap_invalidate_lock (inode->i_mapping); +if (atomic_inc_return (&bdev->bd_fallocate_count) ==1 ) +filemap_invalidate_lock (inode->i_mapping);/* Invalidate the page cache, including dirty pages. */ error =truncate_bdev_range (bdev, file->f_mode, start, end);
@@ -679,7 +680,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start, } fail: -filemap_invalidate_unlock (inode->i_mapping); +if (!atomic_dec_return (&bdev->bd_fallocate_count)) +filemap_invalidate_unlock (inode->i_mapping);return error; }
include/linux/blk_types.h +2 -0
@@ -58,6 +58,8 @@ struct block_device {struct gendisk * bd_disk;struct request_queue * bd_queue; +atomic_t bd_fallocate_count; +/* The counter of freeze processes */ int bd_fsfreeze_count;/* Mutex for freeze */
Signature · 1 line
2.31.1
Quoted 10 lines · Ming Lei
Commit
f278eb3d8178("block: hold ->invalidate_lock inblkdev_fallocate") adds ->invalidate_lock inblkdev_fallocate()for preventing stale data from being read duringfallocate().However, the side effect is that
blkdev_fallocate()becomes serialized sinceblkdev_fallocate()always blocks.Add one atomic fallocate counter for allowing
blkdev_fallocate()to be run concurrently so that discard/write_zero bios from differentfallocate()can be submitted in parallel.
Please consider adding a "Fixes:" tag since otherwise this patch won't be integrated automatically into any of the stable kernel trees.
Thanks,
Bart.
Quoted 7 lines · Ming Lei
Commit
f278eb3d8178("block: hold ->invalidate_lock inblkdev_fallocate") adds ->invalidate_lock inblkdev_fallocate()for preventing stale data from being read duringfallocate().However, the side effect is that
blkdev_fallocate()becomes serialized sinceblkdev_fallocate()always blocks.
Quoted 6 lines
Add one atomic fallocate counter for allowing
blkdev_fallocate()to be run concurrently so that discard/write_zero bios from differentfallocate()can be submitted in parallel.Cc Jan Kara <[email protected]>Signed-off-by Ming Lei <[email protected]>
Couple of points:
* What about blk_ioctl_zeroout() or blk_ioctl_discard()?
* This way processes calling discard / zeroout can easily starve process wanting to do read which does not seem ideal. Somewhat related is that I know that Christoph wanted to modify how we use filemap_invalidate_lock() so that huge zeroouts performed by actually writing zeros will not block readers for ages and this seems to be going in somewhat opposite direction favoring writers even more.
* I suspect this is going to upset lockdep (and lock owner tracking for the purposes of spinning) pretty badly because the lock owner may be returning to userspace without unlocking the lock.
Honza
Quoted 45 lines
--- block/fops.c | 6 ++++-- include/linux/blk_types.h | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-)block/fops.c +4
-2@@ -651,7 +651,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,if ((start | len) & (bdev_logical_block_size (bdev) -1 ))return -EINVAL ; -filemap_invalidate_lock (inode->i_mapping); +if (atomic_inc_return (&bdev->bd_fallocate_count) ==1 ) +filemap_invalidate_lock (inode->i_mapping);/* Invalidate the page cache, including dirty pages. */ error =truncate_bdev_range (bdev, file->f_mode, start, end);@@ -679,7 +680,8 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start, } fail: -filemap_invalidate_unlock (inode->i_mapping); +if (!atomic_dec_return (&bdev->bd_fallocate_count)) +filemap_invalidate_unlock (inode->i_mapping);return error; }include/linux/blk_types.h +2
-0@@ -58,6 +58,8 @@ struct block_device {struct gendisk * bd_disk;struct request_queue * bd_queue; +atomic_t bd_fallocate_count; +/* The counter of freeze processes */ int bd_fsfreeze_count;/* Mutex for freeze */ Signature · 1 line
2.31.1
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 9 lines · Jan Kara
* This way processes calling discard / zeroout can easily starve process wanting to do read which does not seem ideal. Somewhat related is that I know that Christoph wanted to modify how we use
filemap_invalidate_lock()so that huge zeroouts performed by actually writing zeros will not block readers for ages and this seems to be going in somewhat opposite direction favoring writers even more.
* I suspect this is going to upset lockdep (and lock owner tracking for the purposes of spinning) pretty badly because the lock owner may be returning to userspace without unlocking the lock.
Yes to all of the above. I think we need to just pick a smaller defauly zeroout size (and make that tunable) to avoid holding the lock over really long operations. Or turn the invalidate lock into a range lock..