btrfs: Remove BTRFS_INODE_IN_DELALLOC_LIST flag
PATCH 1/1 btrfs: Remove BTRFS_INODE_IN_DELALLOC_LIST flag
The flag simply replicates whether btrfs_inode::delallocs_inodes list is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called under btrfs_root::delalloc_lock.
--- fs/btrfs/btrfs_inode.h | 1 - fs/btrfs/inode.c | 11 ++--------- 2 files changed, 2 insertions(+), 10 deletions(-)
fs/btrfs/btrfs_inode.h +0 -1
@@ -27,7 +27,6 @@ enum {BTRFS_INODE_HAS_ASYNC_EXTENT ,BTRFS_INODE_NEEDS_FULL_SYNC ,BTRFS_INODE_COPY_EVERYTHING , -BTRFS_INODE_IN_DELALLOC_LIST ,BTRFS_INODE_HAS_PROPS , BTRFS_INODE_SNAPSHOT_FLUSH, };
fs/btrfs/inode.c +2 -9
@@ -1865,8 +1865,6 @@ static void btrfs_add_delalloc_inodes(struct btrfs_root *root,if (list_empty (&BTRFS_I (inode)->delalloc_inodes)) {list_add_tail (&BTRFS_I (inode)->delalloc_inodes, &root->delalloc_inodes); -set_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &BTRFS_I (inode)->runtime_flags); root->nr_delalloc_inodes++;if (root->nr_delalloc_inodes ==1 ) {spin_lock (&fs_info->delalloc_root_lock);
@@ -1887,8 +1885,6 @@ void __btrfs_del_delalloc_inode(struct btrfs_root *root,if (!list_empty (&inode->delalloc_inodes)) {list_del_init (&inode->delalloc_inodes); -clear_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &inode->runtime_flags); root->nr_delalloc_inodes--;if (!root->nr_delalloc_inodes) {ASSERT (list_empty (&root->delalloc_inodes));
@@ -1944,8 +1940,7 @@ void btrfs_set_delalloc_extent(struct inode *inode, struct extent_state *state,BTRFS_I (inode)->delalloc_bytes += len;if (*bits &EXTENT_DEFRAG )BTRFS_I (inode)->defrag_bytes += len; -if (do_list && !test_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &BTRFS_I (inode)->runtime_flags)) +if (do_list)btrfs_add_delalloc_inodes (root, inode);spin_unlock (&BTRFS_I (inode)->lock); }
@@ -2014,9 +2009,7 @@ void btrfs_clear_delalloc_extent(struct inode *vfs_inode, fs_info->delalloc_batch);spin_lock (&inode->lock); inode->delalloc_bytes -= len; -if (do_list && inode->delalloc_bytes ==0 && -test_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &inode->runtime_flags)) +if (do_list && inode->delalloc_bytes ==0 )btrfs_del_delalloc_inode (root, inode);spin_unlock (&inode->lock); }
--
2.17.1
Quoted 6 lines · Nikolay Borisov
The flag simply replicates whether
btrfs_inode::delallocs_inodeslist is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called underbtrfs_root::delalloc_lock.Signed-off-by Nikolay Borisov <[email protected]>
Nice. Added to misc-next, thanks.
Quoted 5 lines · Nikolay Borisov
The flag simply replicates whether
btrfs_inode::delallocs_inodeslist is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called underbtrfs_root::delalloc_lock.
The flag is there to avoid taking the root's delalloc_lock spinlock everytime a range is marked for delalloc for any inode of the subvolume.
Have you measured performance with very high concurrency of buffered writes against files in the same subvolume?
Thanks.
Quoted 65 lines
Signed-off-by Nikolay Borisov <[email protected]>--- fs/btrfs/btrfs_inode.h | 1 - fs/btrfs/inode.c | 11 ++--------- 2 files changed, 2 insertions(+), 10 deletions(-)fs/btrfs/btrfs_inode.h +0
-1@@ -27,7 +27,6 @@ enum {BTRFS_INODE_HAS_ASYNC_EXTENT ,BTRFS_INODE_NEEDS_FULL_SYNC ,BTRFS_INODE_COPY_EVERYTHING , -BTRFS_INODE_IN_DELALLOC_LIST ,BTRFS_INODE_HAS_PROPS , BTRFS_INODE_SNAPSHOT_FLUSH, };fs/btrfs/inode.c +2
-10@@ -1865,8 +1865,6 @@ static void btrfs_add_delalloc_inodes(struct btrfs_root *root,if (list_empty (&BTRFS_I (inode)->delalloc_inodes)) {list_add_tail (&BTRFS_I (inode)->delalloc_inodes, &root->delalloc_inodes); -set_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &BTRFS_I (inode)->runtime_flags); root->nr_delalloc_inodes++;if (root->nr_delalloc_inodes ==1 ) {spin_lock (&fs_info->delalloc_root_lock);@@ -1887,8 +1885,6 @@ void __btrfs_del_delalloc_inode(struct btrfs_root *root,if (!list_empty (&inode->delalloc_inodes)) {list_del_init (&inode->delalloc_inodes); -clear_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &inode->runtime_flags); root->nr_delalloc_inodes--;if (!root->nr_delalloc_inodes) {ASSERT (list_empty (&root->delalloc_inodes));@@ -1944,8 +1940,7 @@ void btrfs_set_delalloc_extent(struct inode *inode, struct extent_state *state,BTRFS_I (inode)->delalloc_bytes += len;if (*bits &EXTENT_DEFRAG )BTRFS_I (inode)->defrag_bytes += len; -if (do_list && !test_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &BTRFS_I (inode)->runtime_flags)) +if (do_list)btrfs_add_delalloc_inodes (root, inode);spin_unlock (&BTRFS_I (inode)->lock); }@@ -2014,9 +2009,7 @@ void btrfs_clear_delalloc_extent(struct inode *vfs_inode, fs_info->delalloc_batch);spin_lock (&inode->lock); inode->delalloc_bytes -= len; -if (do_list && inode->delalloc_bytes ==0 && -test_bit (BTRFS_INODE_IN_DELALLOC_LIST , - &inode->runtime_flags)) +if (do_list && inode->delalloc_bytes ==0 )btrfs_del_delalloc_inode (root, inode);spin_unlock (&inode->lock); } --2.17.1
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
Quoted 10 lines · Filipe Manana
Quoted 5 lines · Nikolay Borisov
The flag simply replicates whether
btrfs_inode::delallocs_inodeslist is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called underbtrfs_root::delalloc_lock.The flag is there to avoid taking the root's
delalloc_lockspinlock everytime a range is marked for delalloc for any inode of the subvolume.
I overlooked that not all uses of the bit are under delalloc_lock, which would make it redundant, but both test_bit are inside inode lock, not the delalloc lock.
Quoted 2 lines
Have you measured performance with very high concurrency of buffered writes against files in the same subvolume?
I'll remove the patch for now unless the performance is verified to be ok.
Quoted 7 lines · Filipe Manana
Quoted 5 lines · Nikolay Borisov
The flag simply replicates whether
btrfs_inode::delallocs_inodeslist is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called underbtrfs_root::delalloc_lock.
Quoted 7 lines
The flag is there to avoid taking the root's
delalloc_lockspinlock everytime a range is marked for delalloc for any inode of the subvolume.
Have you measured performance with very high concurrency of buffered writes against files in the same subvolume?Thanks.
I performed the following test on a 16-core VM
(physical cores are 12 on the host):
fio --direct=0 --ioengine=sync --thread --directory=/media/scratch/ --invalidate=1 --group_reporting=1 \
--fallocate=posix --name=RandomWrites-async-64512-4k-4 --new_group --rw=randwrite --size=50m --numjobs=200 \
--bs=4k --fsync_on_close=0 --fallocate=none --end_fsync=0 --filename_format=FioWorkloads.\$jobnum
And here's what /proc/lock_stat report:
With BTRFS_INODE_IN_DELALLOC_LIST:
class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
&root->delalloc_lock: 245 245 0.08 4.14 88.10 0.36 62168 122055 0.05 60.41 32721.41 0.27
Fio output:
WRITE: bw=43.9MiB/s (45.0MB/s), 43.9MiB/s-43.9MiB/s (45.0MB/s-45.0MB/s), io=9.77GiB (10.5GB), run=228044-228044msec
Without BTRFS_INODE_IN_DELALLOC_LIST:
class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
&root->delalloc_lock: 8824 8838 0.05 210.92 3451.03 0.39 2542011 2685019 0.03 301.63 451369.98 0.17
WRITE: bw=33.8MiB/s (35.5MB/s), 33.8MiB/s-33.8MiB/s (35.5MB/s-35.5MB/s), io=9.77GiB (10.5GB), run=295770-295770msec
So yeah, it does have noticeable effect, and massively reduces lock contentions on the delalloc_lock but it increases the critical section, due to the added avg times. But the improvement in performance in terms of throughput and reduced acquires/contentions is indisputable. So yeah, this patch should be dropped.
Thanks for spotting this.