Skip to contentSkip to conversation

btrfs: Remove BTRFS_INODE_IN_DELALLOC_LIST flag

PATCH In review Nikolay Borisov 3 participants 5 messages May 27, 2020 – May 28, 2020 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 btrfs: Remove BTRFS_INODE_IN_DELALLOC_LIST flag

The flag simply replicates whether btrfs_inode::delallocs_inodes list is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called under btrfs_root::delalloc_lock.

---
 fs/btrfs/btrfs_inode.h |  1 -
 fs/btrfs/inode.c       | 11 ++---------
 2 files changed, 2 insertions(+), 10 deletions(-)
fs/btrfs/btrfs_inode.h +0 -1
index aeff56a0e105..da6743c70412 100644
@@ -27,7 +27,6 @@ enum {
 	BTRFS_INODE_HAS_ASYNC_EXTENT,
 	BTRFS_INODE_NEEDS_FULL_SYNC,
 	BTRFS_INODE_COPY_EVERYTHING,
-	BTRFS_INODE_IN_DELALLOC_LIST,
 	BTRFS_INODE_HAS_PROPS,
 	BTRFS_INODE_SNAPSHOT_FLUSH,
 };
fs/btrfs/inode.c +2 -9
index 7d2f6e55a234..3e87a6644e09 100644
@@ -1865,8 +1865,6 @@ static void btrfs_add_delalloc_inodes(struct btrfs_root *root,
 	if (list_empty(&BTRFS_I(inode)->delalloc_inodes)) {
 		list_add_tail(&BTRFS_I(inode)->delalloc_inodes,
 			      &root->delalloc_inodes);
-		set_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-			&BTRFS_I(inode)->runtime_flags);
 		root->nr_delalloc_inodes++;
 		if (root->nr_delalloc_inodes == 1) {
 			spin_lock(&fs_info->delalloc_root_lock);
@@ -1887,8 +1885,6 @@ void __btrfs_del_delalloc_inode(struct btrfs_root *root,
 
 	if (!list_empty(&inode->delalloc_inodes)) {
 		list_del_init(&inode->delalloc_inodes);
-		clear_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-			  &inode->runtime_flags);
 		root->nr_delalloc_inodes--;
 		if (!root->nr_delalloc_inodes) {
 			ASSERT(list_empty(&root->delalloc_inodes));
@@ -1944,8 +1940,7 @@ void btrfs_set_delalloc_extent(struct inode *inode, struct extent_state *state,
 		BTRFS_I(inode)->delalloc_bytes += len;
 		if (*bits & EXTENT_DEFRAG)
 			BTRFS_I(inode)->defrag_bytes += len;
-		if (do_list && !test_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-					 &BTRFS_I(inode)->runtime_flags))
+		if (do_list)
 			btrfs_add_delalloc_inodes(root, inode);
 		spin_unlock(&BTRFS_I(inode)->lock);
 	}
@@ -2014,9 +2009,7 @@ void btrfs_clear_delalloc_extent(struct inode *vfs_inode,
 					 fs_info->delalloc_batch);
 		spin_lock(&inode->lock);
 		inode->delalloc_bytes -= len;
-		if (do_list && inode->delalloc_bytes == 0 &&
-		    test_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-					&inode->runtime_flags))
+		if (do_list && inode->delalloc_bytes == 0)
 			btrfs_del_delalloc_inode(root, inode);
 		spin_unlock(&inode->lock);
 	}

--
2.17.1

#1 lore raw
Quoted 6 lines · Nikolay Borisov

The flag simply replicates whether btrfs_inode::delallocs_inodes list is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called under btrfs_root::delalloc_lock.

Nice. Added to misc-next, thanks.

#2 lore raw
In reply to Nikolay Borisov
Quoted 5 lines · Nikolay Borisov

The flag simply replicates whether btrfs_inode::delallocs_inodes list is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called under btrfs_root::delalloc_lock.

The flag is there to avoid taking the root's delalloc_lock spinlock everytime a range is marked for delalloc for any inode of the subvolume.
Have you measured performance with very high concurrency of buffered writes against files in the same subvolume?

Thanks.

Quoted 65 lines
---
 fs/btrfs/btrfs_inode.h |  1 -
 fs/btrfs/inode.c       | 11 ++---------
 2 files changed, 2 insertions(+), 10 deletions(-)
fs/btrfs/btrfs_inode.h +0 -1
index aeff56a0e105..da6743c70412 100644
@@ -27,7 +27,6 @@ enum {
        BTRFS_INODE_HAS_ASYNC_EXTENT,
        BTRFS_INODE_NEEDS_FULL_SYNC,
        BTRFS_INODE_COPY_EVERYTHING,
-       BTRFS_INODE_IN_DELALLOC_LIST,
        BTRFS_INODE_HAS_PROPS,
        BTRFS_INODE_SNAPSHOT_FLUSH,
 };
fs/btrfs/inode.c +2 -10
index 7d2f6e55a234..3e87a6644e09 100644
@@ -1865,8 +1865,6 @@ static void btrfs_add_delalloc_inodes(struct btrfs_root *root,
        if (list_empty(&BTRFS_I(inode)->delalloc_inodes)) {
                list_add_tail(&BTRFS_I(inode)->delalloc_inodes,
                              &root->delalloc_inodes);
-               set_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-                       &BTRFS_I(inode)->runtime_flags);
                root->nr_delalloc_inodes++;
                if (root->nr_delalloc_inodes == 1) {
                        spin_lock(&fs_info->delalloc_root_lock);
@@ -1887,8 +1885,6 @@ void __btrfs_del_delalloc_inode(struct btrfs_root *root,
 
        if (!list_empty(&inode->delalloc_inodes)) {
                list_del_init(&inode->delalloc_inodes);
-               clear_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-                         &inode->runtime_flags);
                root->nr_delalloc_inodes--;
                if (!root->nr_delalloc_inodes) {
                        ASSERT(list_empty(&root->delalloc_inodes));
@@ -1944,8 +1940,7 @@ void btrfs_set_delalloc_extent(struct inode *inode, struct extent_state *state,
                BTRFS_I(inode)->delalloc_bytes += len;
                if (*bits & EXTENT_DEFRAG)
                        BTRFS_I(inode)->defrag_bytes += len;
-               if (do_list && !test_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-                                        &BTRFS_I(inode)->runtime_flags))
+               if (do_list)
                        btrfs_add_delalloc_inodes(root, inode);
                spin_unlock(&BTRFS_I(inode)->lock);
        }
@@ -2014,9 +2009,7 @@ void btrfs_clear_delalloc_extent(struct inode *vfs_inode,
                                         fs_info->delalloc_batch);
                spin_lock(&inode->lock);
                inode->delalloc_bytes -= len;
-               if (do_list && inode->delalloc_bytes == 0 &&
-                   test_bit(BTRFS_INODE_IN_DELALLOC_LIST,
-                                       &inode->runtime_flags))
+               if (do_list && inode->delalloc_bytes == 0)
                        btrfs_del_delalloc_inode(root, inode);
                spin_unlock(&inode->lock);
        }
--

2.17.1

Signature · 3 lines
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”
#3 lore raw
Quoted 10 lines · Filipe Manana
Quoted 5 lines · Nikolay Borisov

The flag simply replicates whether btrfs_inode::delallocs_inodes list is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called under btrfs_root::delalloc_lock.

The flag is there to avoid taking the root's delalloc_lock spinlock everytime a range is marked for delalloc for any inode of the subvolume.

I overlooked that not all uses of the bit are under delalloc_lock, which would make it redundant, but both test_bit are inside inode lock, not the delalloc lock.

Quoted 2 lines

Have you measured performance with very high concurrency of buffered writes against files in the same subvolume?

I'll remove the patch for now unless the performance is verified to be ok.

#4 lore raw
In reply to Filipe Manana
Quoted 7 lines · Filipe Manana
Quoted 5 lines · Nikolay Borisov

The flag simply replicates whether btrfs_inode::delallocs_inodes list is empty or not. Just defer this check to the list management functions (btrfs_add_delalloc_inodes/__btrfs_del_delalloc_inode) which are always called under btrfs_root::delalloc_lock.

Quoted 7 lines

The flag is there to avoid taking the root's delalloc_lock spinlock everytime a range is marked for delalloc for any inode of the subvolume.
Have you measured performance with very high concurrency of buffered writes against files in the same subvolume?

Thanks.

I performed the following test on a 16-core VM
(physical cores are 12 on the host):

fio --direct=0 --ioengine=sync --thread --directory=/media/scratch/ --invalidate=1 --group_reporting=1 \
--fallocate=posix --name=RandomWrites-async-64512-4k-4 --new_group --rw=randwrite --size=50m --numjobs=200 \
--bs=4k --fsync_on_close=0 --fallocate=none --end_fsync=0 --filename_format=FioWorkloads.\$jobnum

And here's what /proc/lock_stat report:

With BTRFS_INODE_IN_DELALLOC_LIST:

class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
&root->delalloc_lock: 245 245 0.08 4.14 88.10 0.36 62168 122055 0.05 60.41 32721.41 0.27

Fio output:
WRITE: bw=43.9MiB/s (45.0MB/s), 43.9MiB/s-43.9MiB/s (45.0MB/s-45.0MB/s), io=9.77GiB (10.5GB), run=228044-228044msec

Without BTRFS_INODE_IN_DELALLOC_LIST:
class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
&root->delalloc_lock: 8824 8838 0.05 210.92 3451.03 0.39 2542011 2685019 0.03 301.63 451369.98 0.17

WRITE: bw=33.8MiB/s (35.5MB/s), 33.8MiB/s-33.8MiB/s (35.5MB/s-35.5MB/s), io=9.77GiB (10.5GB), run=295770-295770msec

So yeah, it does have noticeable effect, and massively reduces lock contentions on the delalloc_lock but it increases the critical section, due to the added avg times. But the improvement in performance in terms of throughput and reduced acquires/contentions is indisputable. So yeah, this patch should be dropped.

Thanks for spotting this.