Skip to contentSkip to conversation

improve the block time during the transaction commit

PATCH In review Miao Xie 4 participants 33 messages May 15, 2013 – May 17, 2013 Btrfs file system

lore
Share & tools
lore

Miao Xie

author
#0 lore raw

cover improve the block time during the transaction commit

This patchset improve the problem that the transaction may be blocked for a long time when it is being committed if there is heavy I/O.

In this patchset,
- 0001-0005, 0007, 0011-0012 are random fix or code cleanup patch.
- 0006, 0008-0010 introduce per-subvolume delalloc inode list and ordered extent list, which can reduce the flush time when we create snapshots.
- 0013-0016 improve the block time during the transaction commit by removing the while loop at the beginning of the transaction commit.
- 0017 improves the readability of the code.

Miao Xie (17):
Btrfs: fix accessing a freed tree root
Btrfs: fix unprotected root node of the subvolume's inode rb-tree
Btrfs: pause the space balance when remounting to R/O
Btrfs: remove BUG_ON() in btrfs_read_fs_tree_no_radix()
Btrfs: cleanup the similar code of the fs root read
Btrfs: introduce grab/put functions for the root of the fs/file tree
Btrfs: don't invoke btrfs_invalidate_inodes() in the spin lock context
Btrfs: introduce per-subvolume delalloc inode list
Btrfs: introduce per-subvolume ordered extent list
Btrfs: just flush the delalloc inodes in the source tree before snapshot creation
Btrfs: cleanup unnecessary assignment when cleaning up all the residual transaction
Btrfs: remove the code for the impossible case in cleanup_transaction()
Btrfs: don't wait for all the writers circularly during the transaction commit
Btrfs: don't flush the delalloc inodes in the while loop if flushoncommit is set
Btrfs: remove unnecessary varient ->num_joined in btrfs_transaction structure
Btrfs: remove the time check in btrfs_commit_transaction()
Btrfs: make the state of the transaction more readable

 fs/btrfs/ctree.h        |  55 +++++--
 fs/btrfs/dev-replace.c  |   6 +-
 fs/btrfs/disk-io.c      | 425 +++++++++++++++++++++++++++---------------------
 fs/btrfs/disk-io.h      |  32 +++-
 fs/btrfs/extent-tree.c  |  20 +--
 fs/btrfs/inode.c        | 180 ++++++++++++++------
 fs/btrfs/ioctl.c        |   6 +
 fs/btrfs/ordered-data.c | 109 +++++++++----
 fs/btrfs/ordered-data.h |   2 +
 fs/btrfs/relocation.c   |   9 +-
 fs/btrfs/root-tree.c    | 170 ++++++-------------
 fs/btrfs/super.c        |   3 +-
 fs/btrfs/transaction.c  | 271 ++++++++++++++++--------------
 fs/btrfs/transaction.h  |  49 ++++--
 fs/btrfs/tree-log.c     |   3 +-
 fs/btrfs/volumes.c      |  13 +-
 fs/btrfs/volumes.h      |   1 +
 17 files changed, 791 insertions(+), 563 deletions(-)
Signature · 1 line
1.8.1.4
author
#1 lore raw

PATCH 1/17 Btrfs: fix accessing a freed tree root

inode_tree_del() will move the tree root into the dead root list, and then the tree will be destroyed by the cleaner. So if we remove the delayed node which is cached in the inode after inode_tree_del(), we may access a freed tree root. Fix it.

---
 fs/btrfs/inode.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/btrfs/inode.c +1 -1
index 1669c3b..7f6e78a 100644
@@ -4723,6 +4723,7 @@ void btrfs_evict_inode(struct inode *inode)
 	btrfs_end_transaction(trans, root);
 	btrfs_btree_balance_dirty(root);
 no_delete:
+	btrfs_remove_delayed_node(inode);
 	clear_inode(inode);
 	return;
 }
@@ -7978,7 +7979,6 @@ void btrfs_destroy_inode(struct inode *inode)
 	inode_tree_del(inode);
 	btrfs_drop_extent_cache(inode, 0, (u64)-1, 0);
 free:
-	btrfs_remove_delayed_node(inode);
 	call_rcu(&inode->i_rcu, btrfs_i_callback);
 }
 
Signature · 1 line
1.8.1.4
author
#2 lore raw

PATCH 2/17 Btrfs: fix unprotected root node of the subvolume's inode rb-tree

In reply to Miao Xie

The root node of the rb-tree may be changed, so we should get it under the lock. Fix it.

---
 fs/btrfs/inode.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
fs/btrfs/inode.c +3 -4
index 7f6e78a..bf5c399 100644
@@ -4839,14 +4839,13 @@ static void inode_tree_add(struct inode *inode)
 	struct rb_node **p;
 	struct rb_node *parent;
 	u64 ino = btrfs_ino(inode);
-again:
-	p = &root->inode_tree.rb_node;
-	parent = NULL;
 
 	if (inode_unhashed(inode))
 		return;
-
+again:
+	parent = NULL;
 	spin_lock(&root->inode_lock);
+	p = &root->inode_tree.rb_node;
 	while (*p) {
 		parent = *p;
 		entry = rb_entry(parent, struct btrfs_inode, rb_node);
Signature · 1 line
1.8.1.4
author
#3 lore raw

PATCH 3/17 Btrfs: pause the space balance when remounting to R/O

In reply to Miao Xie
---
 fs/btrfs/super.c | 1 +
 1 file changed, 1 insertion(+)
fs/btrfs/super.c +1 -0
index a4807ce..f0857e0 100644
@@ -1263,6 +1263,7 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data)
 
 		btrfs_dev_replace_suspend_for_unmount(fs_info);
 		btrfs_scrub_cancel(fs_info);
+		btrfs_pause_balance(fs_info);
 
 		ret = btrfs_commit_super(root);
 		if (ret)
Signature · 1 line
1.8.1.4
author
#4 lore raw

PATCH 4/17 Btrfs: remove BUG_ON() in btrfs_read_fs_tree_no_radix()

In reply to Miao Xie

We have checked if ->node is NULL or not, so it is unnecessary to use BUG_ON() to check again. Remove it.

---
 fs/btrfs/disk-io.c | 1 -
 1 file changed, 1 deletion(-)
fs/btrfs/disk-io.c +0 -1
index 2a9ae38..8c1e4fb 100644
@@ -1513,7 +1513,6 @@ struct btrfs_root *btrfs_read_fs_root_no_radix(struct btrfs_root *tree_root,
 	}
 
 	root->commit_root = btrfs_root_node(root);
-	BUG_ON(!root->node); /* -ENOMEM */
 out:
 	if (location->objectid != BTRFS_TREE_LOG_OBJECTID) {
 		root->ref_cows = 1;
Signature · 1 line
1.8.1.4
author
#5 lore raw

PATCH 5/17 Btrfs: cleanup the similar code of the fs root read

In reply to Miao Xie

There are several functions whose code is similar, such as
btrfs_find_last_root() btrfs_read_fs_root_no_radix()

Besides that, some functions are invoked twice, it is unnecessary, for example, we are sure that all roots which is found in
btrfs_find_orphan_roots()
have their orphan items, so it is unnecessary to check the orphan item again.

So cleanup it.

---
 fs/btrfs/ctree.h       |   6 +-
 fs/btrfs/disk-io.c     | 282 +++++++++++++++++++++++++------------------------
 fs/btrfs/disk-io.h     |  11 +-
 fs/btrfs/extent-tree.c |   6 +-
 fs/btrfs/relocation.c  |   5 +-
 fs/btrfs/root-tree.c   | 170 ++++++++++-------------------
 fs/btrfs/tree-log.c    |   3 +-
 fs/btrfs/volumes.c     |  13 ++-
 fs/btrfs/volumes.h     |   1 +
 9 files changed, 228 insertions(+), 269 deletions(-)
fs/btrfs/ctree.h +3 -3
index f9a48fd..845b77f 100644
@@ -3369,9 +3369,9 @@ int __must_check btrfs_update_root(struct btrfs_trans_handle *trans,
 				   struct btrfs_root_item *item);
 void btrfs_read_root_item(struct extent_buffer *eb, int slot,
 			  struct btrfs_root_item *item);
-int btrfs_find_last_root(struct btrfs_root *root, u64 objectid, struct
-			 btrfs_root_item *item, struct btrfs_key *key);
-int btrfs_find_dead_roots(struct btrfs_root *root, u64 objectid);
+int btrfs_find_root(struct btrfs_root *root, struct btrfs_key *search_key,
+		    struct btrfs_path *path, struct btrfs_root_item *root_item,
+		    struct btrfs_key *root_key);
 int btrfs_find_orphan_roots(struct btrfs_root *tree_root);
 void btrfs_set_root_node(struct btrfs_root_item *item,
 			 struct extent_buffer *node);
fs/btrfs/disk-io.c +146 -136

410 lines not on this page · raw message

fs/btrfs/disk-io.h +8 -3
index be69ce1..534d583 100644
@@ -63,14 +63,19 @@ struct buffer_head *btrfs_read_dev_super(struct block_device *bdev);
 int btrfs_commit_super(struct btrfs_root *root);
 struct extent_buffer *btrfs_find_tree_block(struct btrfs_root *root,
 					    u64 bytenr, u32 blocksize);
-struct btrfs_root *btrfs_read_fs_root_no_radix(struct btrfs_root *tree_root,
-					       struct btrfs_key *location);
+struct btrfs_root *btrfs_read_fs_root(struct btrfs_root *tree_root,
+				      struct btrfs_key *location);
+int btrfs_init_fs_root(struct btrfs_root *root);
+int btrfs_insert_fs_root(struct btrfs_fs_info *fs_info,
+			 struct btrfs_root *root);
 struct btrfs_root *btrfs_read_fs_root_no_name(struct btrfs_fs_info *fs_info,
 					      struct btrfs_key *location);
 int btrfs_cleanup_fs_roots(struct btrfs_fs_info *fs_info);
 void btrfs_btree_balance_dirty(struct btrfs_root *root);
 void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
-void btrfs_free_fs_root(struct btrfs_fs_info *fs_info, struct btrfs_root *root);
+void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
+				 struct btrfs_root *root);
+void btrfs_free_fs_root(struct btrfs_root *root);
 void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
 int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
 			  int atomic);
fs/btrfs/extent-tree.c +3 -3
index 7f7c7a6..08e42c8 100644
@@ -7441,8 +7441,8 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	}
 
 	if (root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID) {
-		ret = btrfs_find_last_root(tree_root, root->root_key.objectid,
-					   NULL, NULL);
+		ret = btrfs_find_root(tree_root, &root->root_key, path,
+				      NULL, NULL);
 		if (ret < 0) {
 			btrfs_abort_transaction(trans, tree_root, ret);
 			err = ret;
@@ -7459,7 +7459,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	}
 
 	if (root->in_radix) {
-		btrfs_free_fs_root(tree_root->fs_info, root);
+		btrfs_drop_and_free_fs_root(tree_root->fs_info, root);
 	} else {
 		free_extent_buffer(root->node);
 		free_extent_buffer(root->commit_root);
fs/btrfs/relocation.c +2 -3
index 704a1b8..baaa66b 100644
@@ -1355,8 +1355,7 @@ static struct btrfs_root *create_reloc_root(struct btrfs_trans_handle *trans,
 	BUG_ON(ret);
 	kfree(root_item);
 
-	reloc_root = btrfs_read_fs_root_no_radix(root->fs_info->tree_root,
-						 &root_key);
+	reloc_root = btrfs_read_fs_root(root->fs_info->tree_root, &root_key);
 	BUG_ON(IS_ERR(reloc_root));
 	reloc_root->last_trans = trans->transid;
 	return reloc_root;
@@ -4271,7 +4270,7 @@ int btrfs_recover_relocation(struct btrfs_root *root)
 		    key.type != BTRFS_ROOT_ITEM_KEY)
 			break;
 
-		reloc_root = btrfs_read_fs_root_no_radix(root, &key);
+		reloc_root = btrfs_read_fs_root(root, &key);
 		if (IS_ERR(reloc_root)) {
 			err = PTR_ERR(reloc_root);
 			goto out;
fs/btrfs/root-tree.c +53 -117

211 lines not on this page · raw message

fs/btrfs/tree-log.c +1 -2
index c276ac9..a59724e 100644
@@ -4016,8 +4016,7 @@ again:
 		if (found_key.objectid != BTRFS_TREE_LOG_OBJECTID)
 			break;
 
-		log = btrfs_read_fs_root_no_radix(log_root_tree,
-						  &found_key);
+		log = btrfs_read_fs_root(log_root_tree, &found_key);
 		if (IS_ERR(log)) {
 			ret = PTR_ERR(log);
 			btrfs_error(fs_info, ret,
fs/btrfs/volumes.c +11 -2
index a191bac..746d092 100644
@@ -5397,7 +5397,6 @@ static struct btrfs_device *add_missing_dev(struct btrfs_root *root,
 		return NULL;
 	list_add(&device->dev_list,
 		 &fs_devices->devices);
-	device->dev_root = root->fs_info->dev_root;
 	device->devid = devid;
 	device->work.func = pending_bios_fn;
 	device->fs_devices = fs_devices;
@@ -5623,7 +5622,6 @@ static int read_one_dev(struct btrfs_root *root,
 	}
 
 	fill_device_from_item(leaf, dev_item, device);
-	device->dev_root = root->fs_info->dev_root;
 	device->in_fs_metadata = 1;
 	if (device->writeable && !device->is_tgtdev_for_dev_replace) {
 		device->fs_devices->total_rw_bytes += device->total_bytes;
@@ -5781,6 +5779,17 @@ error:
 	return ret;
 }
 
+void btrfs_init_devices_late(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
+	struct btrfs_device *device;
+
+	mutex_lock(&fs_devices->device_list_mutex);
+	list_for_each_entry(device, &fs_devices->devices, dev_list)
+		device->dev_root = fs_info->dev_root;
+	mutex_unlock(&fs_devices->device_list_mutex);
+}
+
 static void __btrfs_reset_dev_stats(struct btrfs_device *dev)
 {
 	int i;
fs/btrfs/volumes.h +1 -0
index 845ccbb..7772c2e 100644
@@ -301,6 +301,7 @@ int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
 void btrfs_dev_stat_inc_and_print(struct btrfs_device *dev, int index);
 int btrfs_get_dev_stats(struct btrfs_root *root,
 			struct btrfs_ioctl_get_dev_stats *stats);
+void btrfs_init_devices_late(struct btrfs_fs_info *fs_info);
 int btrfs_init_dev_stats(struct btrfs_fs_info *fs_info);
 int btrfs_run_dev_stats(struct btrfs_trans_handle *trans,
 			struct btrfs_fs_info *fs_info);
Signature · 1 line
1.8.1.4
author
#6 lore raw

PATCH 6/17 Btrfs: introduce grab/put functions for the root of the fs/file tree

In reply to Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     |  5 +++--
 fs/btrfs/disk-io.h     | 21 +++++++++++++++++++++
 fs/btrfs/extent-tree.c |  2 +-
 4 files changed, 26 insertions(+), 3 deletions(-)
fs/btrfs/ctree.h +1 -0
index 845b77f..958ce6c 100644
@@ -1739,6 +1739,7 @@ struct btrfs_root {
 	int force_cow;
 
 	spinlock_t root_item_lock;
+	atomic_t refs;
 };
 
 struct btrfs_ioctl_defrag_range_args {
fs/btrfs/disk-io.c +3 -2
index 42d6ba2..642c861 100644
@@ -1216,6 +1216,7 @@ static void __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
 	atomic_set(&root->log_writers, 0);
 	atomic_set(&root->log_batch, 0);
 	atomic_set(&root->orphan_inodes, 0);
+	atomic_set(&root->refs, 1);
 	root->log_transid = 0;
 	root->last_log_commit = 0;
 	extent_io_tree_init(&root->dirty_log_pages,
@@ -2049,7 +2050,7 @@ static void del_fs_roots(struct btrfs_fs_info *fs_info)
 		} else {
 			free_extent_buffer(gang[0]->node);
 			free_extent_buffer(gang[0]->commit_root);
-			kfree(gang[0]);
+			btrfs_put_fs_root(gang[0]);
 		}
 	}
 
@@ -3415,7 +3416,7 @@ static void free_fs_root(struct btrfs_root *root)
 	kfree(root->free_ino_ctl);
 	kfree(root->free_ino_pinned);
 	kfree(root->name);
-	kfree(root);
+	btrfs_put_fs_root(root);
 }
 
 void btrfs_free_fs_root(struct btrfs_root *root)
fs/btrfs/disk-io.h +21 -0
index 534d583..b71acd6e 100644
@@ -76,6 +76,27 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
 void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
 				 struct btrfs_root *root);
 void btrfs_free_fs_root(struct btrfs_root *root);
+
+/*
+ * This function is used to grab the root, and avoid it is freed when we
+ * access it. But it doesn't ensure that the tree is not dropped.
+ *
+ * If you want to ensure the whole tree is safe, you should use
+ * 	fs_info->subvol_srcu
+ */
+static inline struct btrfs_root *btrfs_grab_fs_root(struct btrfs_root *root)
+{
+	if (atomic_inc_not_zero(&root->refs))
+		return root;
+	return NULL;
+}
+
+static inline void btrfs_put_fs_root(struct btrfs_root *root)
+{
+	if (atomic_dec_and_test(&root->refs))
+		kfree(root);
+}
+
 void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
 int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
 			  int atomic);
fs/btrfs/extent-tree.c +1 -1
index 08e42c8..08f9862 100644
@@ -7463,7 +7463,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	} else {
 		free_extent_buffer(root->node);
 		free_extent_buffer(root->commit_root);
-		kfree(root);
+		btrfs_put_fs_root(root);
 	}
 out_end_trans:
 	btrfs_end_transaction_throttle(trans, tree_root);
Signature · 1 line
1.8.1.4
#7 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 5 lines · Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

I don't think this is necessary, we put 'kfree(root)' because we really need to free them at the very end time, when there should be no inodes linking on the root(we should have cleaned all inodes out from it).

So when we flush delalloc inodes and wait for ordered extents to finish, the root should be valid, otherwise someone is doing wrong things.

And even with this grab_fs_root to avoid freeing root, it's just the root that remains in memory, all its attributes, like root->node, commit_root, root->inode_tree, are already NULL or empty.

thanks,
liubo

Quoted 102 lines
---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     |  5 +++--
 fs/btrfs/disk-io.h     | 21 +++++++++++++++++++++
 fs/btrfs/extent-tree.c |  2 +-
 4 files changed, 26 insertions(+), 3 deletions(-)
fs/btrfs/ctree.h +1 -0
index 845b77f..958ce6c 100644
@@ -1739,6 +1739,7 @@ struct btrfs_root {
 	int force_cow;
 
 	spinlock_t root_item_lock;
+	atomic_t refs;
 };
 
 struct btrfs_ioctl_defrag_range_args {
fs/btrfs/disk-io.c +3 -2
index 42d6ba2..642c861 100644
@@ -1216,6 +1216,7 @@ static void __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
 	atomic_set(&root->log_writers, 0);
 	atomic_set(&root->log_batch, 0);
 	atomic_set(&root->orphan_inodes, 0);
+	atomic_set(&root->refs, 1);
 	root->log_transid = 0;
 	root->last_log_commit = 0;
 	extent_io_tree_init(&root->dirty_log_pages,
@@ -2049,7 +2050,7 @@ static void del_fs_roots(struct btrfs_fs_info *fs_info)
 		} else {
 			free_extent_buffer(gang[0]->node);
 			free_extent_buffer(gang[0]->commit_root);
-			kfree(gang[0]);
+			btrfs_put_fs_root(gang[0]);
 		}
 	}
 
@@ -3415,7 +3416,7 @@ static void free_fs_root(struct btrfs_root *root)
 	kfree(root->free_ino_ctl);
 	kfree(root->free_ino_pinned);
 	kfree(root->name);
-	kfree(root);
+	btrfs_put_fs_root(root);
 }
 
 void btrfs_free_fs_root(struct btrfs_root *root)
fs/btrfs/disk-io.h +21 -0
index 534d583..b71acd6e 100644
@@ -76,6 +76,27 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
 void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
 				 struct btrfs_root *root);
 void btrfs_free_fs_root(struct btrfs_root *root);
+
+/*
+ * This function is used to grab the root, and avoid it is freed when we
+ * access it. But it doesn't ensure that the tree is not dropped.
+ *
+ * If you want to ensure the whole tree is safe, you should use
+ * 	fs_info->subvol_srcu
+ */
+static inline struct btrfs_root *btrfs_grab_fs_root(struct btrfs_root *root)
+{
+	if (atomic_inc_not_zero(&root->refs))
+		return root;
+	return NULL;
+}
+
+static inline void btrfs_put_fs_root(struct btrfs_root *root)
+{
+	if (atomic_dec_and_test(&root->refs))
+		kfree(root);
+}
+
 void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
 int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
 			  int atomic);
fs/btrfs/extent-tree.c +1 -1
index 08e42c8..08f9862 100644
@@ -7463,7 +7463,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	} else {
 		free_extent_buffer(root->node);
 		free_extent_buffer(root->commit_root);
-		kfree(root);
+		btrfs_put_fs_root(root);
 	}
 out_end_trans:
 	btrfs_end_transaction_throttle(trans, tree_root);
Signature · 6 lines
1.8.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#8 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 11 lines · Liu Bo
Quoted 5 lines · Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

I don't think this is necessary, we put 'kfree(root)' because we really need to free them at the very end time, when there should be no inodes linking on the root(we should have cleaned all inodes out from it).

Quoted 6 lines

So when we flush delalloc inodes and wait for ordered extents to finish, the root should be valid, otherwise someone is doing wrong things.

And even with this grab_fs_root to avoid freeing root, it's just the root that remains in memory, all its attributes, like root->node, commit_root, root->inode_tree, are already NULL or empty.

Please consider the case:
Task1 Task2 Cleaner
get the root
flush all delalloc inodes
drop subvolume
iput the last inode
move the root into the dead list
drop subvolume kfree(root)
If Task1 accesses the root now, oops will happen.

I introduced there two functions just to protect the access of the root object, not its attributes, so don't worry about the attributes. (Please see the first sentence of the changelog.)

Thanks
Miao

Quoted 111 lines

thanks,
liubo

Quoted 102 lines
---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     |  5 +++--
 fs/btrfs/disk-io.h     | 21 +++++++++++++++++++++
 fs/btrfs/extent-tree.c |  2 +-
 4 files changed, 26 insertions(+), 3 deletions(-)
fs/btrfs/ctree.h +1 -0
index 845b77f..958ce6c 100644
@@ -1739,6 +1739,7 @@ struct btrfs_root {
 	int force_cow;
 
 	spinlock_t root_item_lock;
+	atomic_t refs;
 };
 
 struct btrfs_ioctl_defrag_range_args {
fs/btrfs/disk-io.c +3 -2
index 42d6ba2..642c861 100644
@@ -1216,6 +1216,7 @@ static void __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
 	atomic_set(&root->log_writers, 0);
 	atomic_set(&root->log_batch, 0);
 	atomic_set(&root->orphan_inodes, 0);
+	atomic_set(&root->refs, 1);
 	root->log_transid = 0;
 	root->last_log_commit = 0;
 	extent_io_tree_init(&root->dirty_log_pages,
@@ -2049,7 +2050,7 @@ static void del_fs_roots(struct btrfs_fs_info *fs_info)
 		} else {
 			free_extent_buffer(gang[0]->node);
 			free_extent_buffer(gang[0]->commit_root);
-			kfree(gang[0]);
+			btrfs_put_fs_root(gang[0]);
 		}
 	}
 
@@ -3415,7 +3416,7 @@ static void free_fs_root(struct btrfs_root *root)
 	kfree(root->free_ino_ctl);
 	kfree(root->free_ino_pinned);
 	kfree(root->name);
-	kfree(root);
+	btrfs_put_fs_root(root);
 }
 
 void btrfs_free_fs_root(struct btrfs_root *root)
fs/btrfs/disk-io.h +21 -0
index 534d583..b71acd6e 100644
@@ -76,6 +76,27 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
 void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
 				 struct btrfs_root *root);
 void btrfs_free_fs_root(struct btrfs_root *root);
+
+/*
+ * This function is used to grab the root, and avoid it is freed when we
+ * access it. But it doesn't ensure that the tree is not dropped.
+ *
+ * If you want to ensure the whole tree is safe, you should use
+ * 	fs_info->subvol_srcu
+ */
+static inline struct btrfs_root *btrfs_grab_fs_root(struct btrfs_root *root)
+{
+	if (atomic_inc_not_zero(&root->refs))
+		return root;
+	return NULL;
+}
+
+static inline void btrfs_put_fs_root(struct btrfs_root *root)
+{
+	if (atomic_dec_and_test(&root->refs))
+		kfree(root);
+}
+
 void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
 int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
 			  int atomic);
fs/btrfs/extent-tree.c +1 -1
index 08e42c8..08f9862 100644
@@ -7463,7 +7463,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	} else {
 		free_extent_buffer(root->node);
 		free_extent_buffer(root->commit_root);
-		kfree(root);
+		btrfs_put_fs_root(root);
 	}
 out_end_trans:
 	btrfs_end_transaction_throttle(trans, tree_root);
Signature · 6 lines
1.8.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 19 lines · Miao Xie
Quoted 11 lines · Liu Bo
Quoted 5 lines · Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

I don't think this is necessary, we put 'kfree(root)' because we really need to free them at the very end time, when there should be no inodes linking on the root(we should have cleaned all inodes out from it).

Quoted 6 lines

So when we flush delalloc inodes and wait for ordered extents to finish, the root should be valid, otherwise someone is doing wrong things.

And even with this grab_fs_root to avoid freeing root, it's just the root that remains in memory, all its attributes, like root->node, commit_root, root->inode_tree, are already NULL or empty.

Quoted 10 lines

Please consider the case:
Task1 Task2 Cleaner
get the root
flush all delalloc inodes
drop subvolume
iput the last inode
move the root into the dead list
drop subvolume kfree(root)
If Task1 accesses the root now, oops will happen.

Then it's task1's fault, why it is not protected by subvol_srcu section when it's possible that someone like task2 sets root's refs to 0?

synchronize_srcu(subvol_srcu) before adding root into dead root list is just for this race case, why do we need another?

thanks,
liubo

Quoted 120 lines

I introduced there two functions just to protect the access of the root object, not its attributes, so don't worry about the attributes. (Please see the first sentence of the changelog.)

Thanks
Miao

Quoted 111 lines

thanks,
liubo

Quoted 102 lines
---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     |  5 +++--
 fs/btrfs/disk-io.h     | 21 +++++++++++++++++++++
 fs/btrfs/extent-tree.c |  2 +-
 4 files changed, 26 insertions(+), 3 deletions(-)
fs/btrfs/ctree.h +1 -0
index 845b77f..958ce6c 100644
@@ -1739,6 +1739,7 @@ struct btrfs_root {
 	int force_cow;
 
 	spinlock_t root_item_lock;
+	atomic_t refs;
 };
 
 struct btrfs_ioctl_defrag_range_args {
fs/btrfs/disk-io.c +3 -2
index 42d6ba2..642c861 100644
@@ -1216,6 +1216,7 @@ static void __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
 	atomic_set(&root->log_writers, 0);
 	atomic_set(&root->log_batch, 0);
 	atomic_set(&root->orphan_inodes, 0);
+	atomic_set(&root->refs, 1);
 	root->log_transid = 0;
 	root->last_log_commit = 0;
 	extent_io_tree_init(&root->dirty_log_pages,
@@ -2049,7 +2050,7 @@ static void del_fs_roots(struct btrfs_fs_info *fs_info)
 		} else {
 			free_extent_buffer(gang[0]->node);
 			free_extent_buffer(gang[0]->commit_root);
-			kfree(gang[0]);
+			btrfs_put_fs_root(gang[0]);
 		}
 	}
 
@@ -3415,7 +3416,7 @@ static void free_fs_root(struct btrfs_root *root)
 	kfree(root->free_ino_ctl);
 	kfree(root->free_ino_pinned);
 	kfree(root->name);
-	kfree(root);
+	btrfs_put_fs_root(root);
 }
 
 void btrfs_free_fs_root(struct btrfs_root *root)
fs/btrfs/disk-io.h +21 -0
index 534d583..b71acd6e 100644
@@ -76,6 +76,27 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
 void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
 				 struct btrfs_root *root);
 void btrfs_free_fs_root(struct btrfs_root *root);
+
+/*
+ * This function is used to grab the root, and avoid it is freed when we
+ * access it. But it doesn't ensure that the tree is not dropped.
+ *
+ * If you want to ensure the whole tree is safe, you should use
+ * 	fs_info->subvol_srcu
+ */
+static inline struct btrfs_root *btrfs_grab_fs_root(struct btrfs_root *root)
+{
+	if (atomic_inc_not_zero(&root->refs))
+		return root;
+	return NULL;
+}
+
+static inline void btrfs_put_fs_root(struct btrfs_root *root)
+{
+	if (atomic_dec_and_test(&root->refs))
+		kfree(root);
+}
+
 void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
 int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
 			  int atomic);
fs/btrfs/extent-tree.c +1 -1
index 08e42c8..08f9862 100644
@@ -7463,7 +7463,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	} else {
 		free_extent_buffer(root->node);
 		free_extent_buffer(root->commit_root);
-		kfree(root);
+		btrfs_put_fs_root(root);
 	}
 out_end_trans:
 	btrfs_end_transaction_throttle(trans, tree_root);
Signature · 6 lines
1.8.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#10 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 31 lines · Liu Bo
Quoted 19 lines · Miao Xie
Quoted 11 lines · Liu Bo
Quoted 5 lines · Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

I don't think this is necessary, we put 'kfree(root)' because we really need to free them at the very end time, when there should be no inodes linking on the root(we should have cleaned all inodes out from it).

Quoted 6 lines

So when we flush delalloc inodes and wait for ordered extents to finish, the root should be valid, otherwise someone is doing wrong things.

And even with this grab_fs_root to avoid freeing root, it's just the root that remains in memory, all its attributes, like root->node, commit_root, root->inode_tree, are already NULL or empty.

Quoted 10 lines

Please consider the case:
Task1 Task2 Cleaner
get the root
flush all delalloc inodes
drop subvolume
iput the last inode
move the root into the dead list
drop subvolume kfree(root)
If Task1 accesses the root now, oops will happen.

Quoted 5 lines

Then it's task1's fault, why it is not protected by subvol_srcu section when it's possible that someone like task2 sets root's refs to 0?

synchronize_srcu(subvol_srcu) before adding root into dead root list is just for this race case, why do we need another?

Please read my changelog.

Miao

Quoted 125 lines

thanks,
liubo

Quoted 120 lines

I introduced there two functions just to protect the access of the root object, not its attributes, so don't worry about the attributes. (Please see the first sentence of the changelog.)

Thanks
Miao

Quoted 111 lines

thanks,
liubo

Quoted 102 lines
---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     |  5 +++--
 fs/btrfs/disk-io.h     | 21 +++++++++++++++++++++
 fs/btrfs/extent-tree.c |  2 +-
 4 files changed, 26 insertions(+), 3 deletions(-)
fs/btrfs/ctree.h +1 -0
index 845b77f..958ce6c 100644
@@ -1739,6 +1739,7 @@ struct btrfs_root {
 	int force_cow;
 
 	spinlock_t root_item_lock;
+	atomic_t refs;
 };
 
 struct btrfs_ioctl_defrag_range_args {
fs/btrfs/disk-io.c +3 -2
index 42d6ba2..642c861 100644
@@ -1216,6 +1216,7 @@ static void __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
 	atomic_set(&root->log_writers, 0);
 	atomic_set(&root->log_batch, 0);
 	atomic_set(&root->orphan_inodes, 0);
+	atomic_set(&root->refs, 1);
 	root->log_transid = 0;
 	root->last_log_commit = 0;
 	extent_io_tree_init(&root->dirty_log_pages,
@@ -2049,7 +2050,7 @@ static void del_fs_roots(struct btrfs_fs_info *fs_info)
 		} else {
 			free_extent_buffer(gang[0]->node);
 			free_extent_buffer(gang[0]->commit_root);
-			kfree(gang[0]);
+			btrfs_put_fs_root(gang[0]);
 		}
 	}
 
@@ -3415,7 +3416,7 @@ static void free_fs_root(struct btrfs_root *root)
 	kfree(root->free_ino_ctl);
 	kfree(root->free_ino_pinned);
 	kfree(root->name);
-	kfree(root);
+	btrfs_put_fs_root(root);
 }
 
 void btrfs_free_fs_root(struct btrfs_root *root)
fs/btrfs/disk-io.h +21 -0
index 534d583..b71acd6e 100644
@@ -76,6 +76,27 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
 void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
 				 struct btrfs_root *root);
 void btrfs_free_fs_root(struct btrfs_root *root);
+
+/*
+ * This function is used to grab the root, and avoid it is freed when we
+ * access it. But it doesn't ensure that the tree is not dropped.
+ *
+ * If you want to ensure the whole tree is safe, you should use
+ * 	fs_info->subvol_srcu
+ */
+static inline struct btrfs_root *btrfs_grab_fs_root(struct btrfs_root *root)
+{
+	if (atomic_inc_not_zero(&root->refs))
+		return root;
+	return NULL;
+}
+
+static inline void btrfs_put_fs_root(struct btrfs_root *root)
+{
+	if (atomic_dec_and_test(&root->refs))
+		kfree(root);
+}
+
 void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
 int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
 			  int atomic);
fs/btrfs/extent-tree.c +1 -1
index 08e42c8..08f9862 100644
@@ -7463,7 +7463,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	} else {
 		free_extent_buffer(root->node);
 		free_extent_buffer(root->commit_root);
-		kfree(root);
+		btrfs_put_fs_root(root);
 	}
 out_end_trans:
 	btrfs_end_transaction_throttle(trans, tree_root);
Signature · 6 lines
1.8.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#11 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 32 lines · Miao Xie
Quoted 31 lines · Liu Bo
Quoted 29 lines · Miao Xie
Quoted 11 lines · Liu Bo
Quoted 5 lines · Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

I don't think this is necessary, we put 'kfree(root)' because we really need to free them at the very end time, when there should be no inodes linking on the root(we should have cleaned all inodes out from it).

Quoted 6 lines

So when we flush delalloc inodes and wait for ordered extents to finish, the root should be valid, otherwise someone is doing wrong things.

And even with this grab_fs_root to avoid freeing root, it's just the root that remains in memory, all its attributes, like root->node, commit_root, root->inode_tree, are already NULL or empty.

Please consider the case:
Task1 Task2 Cleaner
get the root
flush all delalloc inodes
drop subvolume
iput the last inode
move the root into the dead list
drop subvolume kfree(root)
If Task1 accesses the root now, oops will happen.

Quoted 7 lines
Quoted 5 lines

Then it's task1's fault, why it is not protected by subvol_srcu section when it's possible that someone like task2 sets root's refs to 0?

synchronize_srcu(subvol_srcu) before adding root into dead root list is just for this race case, why do we need another?

Please read my changelog.

'The memory reclaim task' in the changelog refers to

	iput
	  -> inode_tree_del

, right?

I don't like special cases, this get/put is different from our usual way:

if (atomic_dec_and_test(refs)) {
	kfree(A->a);
	kfree(A->b);
	kfree(A);
}

According to the pictured case, task1 may also access root->something.

I must say that the patch itself looks harmless, the reason is not good enough.

thanks,
liubo

Quoted 129 lines

Miao

Quoted 125 lines

thanks,
liubo

Quoted 120 lines

I introduced there two functions just to protect the access of the root object, not its attributes, so don't worry about the attributes. (Please see the first sentence of the changelog.)

Thanks
Miao

Quoted 111 lines

thanks,
liubo

Quoted 102 lines
---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     |  5 +++--
 fs/btrfs/disk-io.h     | 21 +++++++++++++++++++++
 fs/btrfs/extent-tree.c |  2 +-
 4 files changed, 26 insertions(+), 3 deletions(-)
fs/btrfs/ctree.h +1 -0
index 845b77f..958ce6c 100644
@@ -1739,6 +1739,7 @@ struct btrfs_root {
 	int force_cow;
 
 	spinlock_t root_item_lock;
+	atomic_t refs;
 };
 
 struct btrfs_ioctl_defrag_range_args {
fs/btrfs/disk-io.c +3 -2
index 42d6ba2..642c861 100644
@@ -1216,6 +1216,7 @@ static void __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
 	atomic_set(&root->log_writers, 0);
 	atomic_set(&root->log_batch, 0);
 	atomic_set(&root->orphan_inodes, 0);
+	atomic_set(&root->refs, 1);
 	root->log_transid = 0;
 	root->last_log_commit = 0;
 	extent_io_tree_init(&root->dirty_log_pages,
@@ -2049,7 +2050,7 @@ static void del_fs_roots(struct btrfs_fs_info *fs_info)
 		} else {
 			free_extent_buffer(gang[0]->node);
 			free_extent_buffer(gang[0]->commit_root);
-			kfree(gang[0]);
+			btrfs_put_fs_root(gang[0]);
 		}
 	}
 
@@ -3415,7 +3416,7 @@ static void free_fs_root(struct btrfs_root *root)
 	kfree(root->free_ino_ctl);
 	kfree(root->free_ino_pinned);
 	kfree(root->name);
-	kfree(root);
+	btrfs_put_fs_root(root);
 }
 
 void btrfs_free_fs_root(struct btrfs_root *root)
fs/btrfs/disk-io.h +21 -0
index 534d583..b71acd6e 100644
@@ -76,6 +76,27 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
 void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
 				 struct btrfs_root *root);
 void btrfs_free_fs_root(struct btrfs_root *root);
+
+/*
+ * This function is used to grab the root, and avoid it is freed when we
+ * access it. But it doesn't ensure that the tree is not dropped.
+ *
+ * If you want to ensure the whole tree is safe, you should use
+ * 	fs_info->subvol_srcu
+ */
+static inline struct btrfs_root *btrfs_grab_fs_root(struct btrfs_root *root)
+{
+	if (atomic_inc_not_zero(&root->refs))
+		return root;
+	return NULL;
+}
+
+static inline void btrfs_put_fs_root(struct btrfs_root *root)
+{
+	if (atomic_dec_and_test(&root->refs))
+		kfree(root);
+}
+
 void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
 int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
 			  int atomic);
fs/btrfs/extent-tree.c +1 -1
index 08e42c8..08f9862 100644
@@ -7463,7 +7463,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
 	} else {
 		free_extent_buffer(root->node);
 		free_extent_buffer(root->commit_root);
-		kfree(root);
+		btrfs_put_fs_root(root);
 	}
 out_end_trans:
 	btrfs_end_transaction_throttle(trans, tree_root);
Signature · 6 lines
1.8.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#12 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 46 lines · Liu Bo
Quoted 32 lines · Miao Xie
Quoted 31 lines · Liu Bo
Quoted 29 lines · Miao Xie
Quoted 11 lines · Liu Bo
Quoted 5 lines · Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

I don't think this is necessary, we put 'kfree(root)' because we really need to free them at the very end time, when there should be no inodes linking on the root(we should have cleaned all inodes out from it).

Quoted 6 lines

So when we flush delalloc inodes and wait for ordered extents to finish, the root should be valid, otherwise someone is doing wrong things.

And even with this grab_fs_root to avoid freeing root, it's just the root that remains in memory, all its attributes, like root->node, commit_root, root->inode_tree, are already NULL or empty.

Please consider the case:
Task1 Task2 Cleaner
get the root
flush all delalloc inodes
drop subvolume
iput the last inode
move the root into the dead list
drop subvolume kfree(root)
If Task1 accesses the root now, oops will happen.

Quoted 7 lines
Quoted 5 lines

Then it's task1's fault, why it is not protected by subvol_srcu section when it's possible that someone like task2 sets root's refs to 0?

synchronize_srcu(subvol_srcu) before adding root into dead root list is just for this race case, why do we need another?

Please read my changelog.

'The memory reclaim task' in the changelog refers to

	iput
	  -> inode_tree_del

, right?

Quoted 8 lines

I don't like special cases, this get/put is different from our usual way:

if (atomic_dec_and_test(refs)) {
	kfree(A->a);
	kfree(A->b);
	kfree(A);
}

According to the pictured case, task1 may also access root->something.

"->something" are the member variants of root object, so the problem you worry about can not happen unless somebody misuse this mechanism in the future. But you can not ascribe the misuse to this patch.

Quoted 1 line

I must say that the patch itself looks harmless, the reason is not good enough.

I don't agree with you.
It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem.

Thanks
Miao

#13 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoting Miao Xie (2013-05-16 03:22:37)

Quoted 5 lines
Quoted 1 line

I must say that the patch itself looks harmless, the reason is not good enough.

I don't agree with you.
It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem.

synchronize_rcu and friends can take a very very long time. I like this patch as a way to avoid them, it's just keeps the whole kernel moving.

-chris

#14 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 11 lines · Chris Mason

Quoting Miao Xie (2013-05-16 03:22:37)

Quoted 5 lines
Quoted 1 line

I must say that the patch itself looks harmless, the reason is not good enough.

I don't agree with you.
It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem.

synchronize_rcu and friends can take a very very long time. I like this patch as a way to avoid them, it's just keeps the whole kernel moving.

-chris

Okay, that teaches me another lesson, thanks Miao :)

thanks,
liubo

#15 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoting Liu Bo (2013-05-16 10:31:39)

Quoted 8 lines · Chris Mason
Quoted 7 lines

Quoting Miao Xie (2013-05-16 03:22:37)

Quoted 5 lines
Quoted 1 line

I must say that the patch itself looks harmless, the reason is not good enough.

I don't agree with you.
It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem.

Quoted 6 lines
Quoted 4 lines

synchronize_rcu and friends can take a very very long time. I like this patch as a way to avoid them, it's just keeps the whole kernel moving.

-chris

Okay, that teaches me another lesson, thanks Miao :)

Actually using the rcu api isn't a huge impact. It's just the synchronize_rcu variants that should be avoided ;)

-chris

#16 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 12 lines · Chris Mason

Quoting Liu Bo (2013-05-16 10:31:39)

Quoted 11 lines · Chris Mason
Quoted 10 lines

Quoting Miao Xie (2013-05-16 03:22:37)

Quoted 5 lines
Quoted 1 line

I must say that the patch itself looks harmless, the reason is not good enough.

I don't agree with you.
It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem.

synchronize_rcu and friends can take a very very long time. I like this patch as a way to avoid them, it's just keeps the whole kernel moving.

Quoted 9 lines
Quoted 3 lines
Quoted 1 line

-chris

Okay, that teaches me another lesson, thanks Miao :)

Actually using the rcu api isn't a huge impact. It's just the synchronize_rcu variants that should be avoided ;)

-chris

Now that it's so slow, I wonder why not use call_srcu() instead?

thanks,
liubo

author
#17 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

Quoted 17 lines · Liu Bo
Quoted 16 lines · Chris Mason

Quoting Liu Bo (2013-05-16 10:31:39)

Quoted 14 lines · Chris Mason
Quoted 11 lines

Quoting Miao Xie (2013-05-16 03:22:37)

Quoted 5 lines
Quoted 1 line

I must say that the patch itself looks harmless, the reason is not good enough.

I don't agree with you.
It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem.

synchronize_rcu and friends can take a very very long time. I like this patch as a way to avoid them, it's just keeps the whole kernel moving.

-chris

Okay, that teaches me another lesson, thanks Miao :)

Quoted 7 lines
Quoted 5 lines

Actually using the rcu api isn't a huge impact. It's just the synchronize_rcu variants that should be avoided ;)

-chris

Now that it's so slow, I wonder why not use call_srcu() instead?

I have considered this approach, but there is a problem that someone may insert a new inode after we call call_srcu(). So I had to make this patch as a interim solution. My next work is to remove synchronize_srcu() from tree_del_inode().

Thanks
Miao

#18 lore raw

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

In reply to Miao Xie
Quoted 60 lines · Miao Xie
Quoted 46 lines · Liu Bo
Quoted 32 lines · Miao Xie
Quoted 31 lines · Liu Bo
Quoted 29 lines · Miao Xie
Quoted 11 lines · Liu Bo
Quoted 5 lines · Miao Xie

The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes synchronize_srcu().

I don't think this is necessary, we put 'kfree(root)' because we really need to free them at the very end time, when there should be no inodes linking on the root(we should have cleaned all inodes out from it).

Quoted 6 lines

So when we flush delalloc inodes and wait for ordered extents to finish, the root should be valid, otherwise someone is doing wrong things.

And even with this grab_fs_root to avoid freeing root, it's just the root that remains in memory, all its attributes, like root->node, commit_root, root->inode_tree, are already NULL or empty.

Please consider the case:
Task1 Task2 Cleaner
get the root
flush all delalloc inodes
drop subvolume
iput the last inode
move the root into the dead list
drop subvolume kfree(root)
If Task1 accesses the root now, oops will happen.

Quoted 7 lines
Quoted 5 lines

Then it's task1's fault, why it is not protected by subvol_srcu section when it's possible that someone like task2 sets root's refs to 0?

synchronize_srcu(subvol_srcu) before adding root into dead root list is just for this race case, why do we need another?

Please read my changelog.

'The memory reclaim task' in the changelog refers to

	iput
	  -> inode_tree_del

, right?

Quoted 8 lines

I don't like special cases, this get/put is different from our usual way:

if (atomic_dec_and_test(refs)) {
	kfree(A->a);
	kfree(A->b);
	kfree(A);
}

According to the pictured case, task1 may also access root->something.

"->something" are the member variants of root object, so the problem you worry about can not happen unless somebody misuse this mechanism in the future. But you can not ascribe the misuse to this patch.

Quoted 5 lines
Quoted 1 line

I must say that the patch itself looks harmless, the reason is not good enough.

I don't agree with you.
It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem.

So the real question is why do you think it will be blocked for a long time by subvol_srcu? Have you already noticed btrfs acting like this somehow?

I played with subvol_srcu for defrag bug some time ago, on the read side we use subvol_srcu as

	index = srcu_read_lock(&fs_info->subvol_srcu);
	get root and check btrfs_root_refs();
	inode = btrfs_iget(inode);
	srcu_read_unlock(&fs_info->subvol_srcu, index);

and yes if we've deleted the subvol/snap, this section will nicely bail out after checking btrfs_root_refs().

Even if we now have thousands of this kind of srcu section, would that be a long time?

thanks,
liubo

author
#19 lore raw

PATCH 7/17 Btrfs: don't invoke btrfs_invalidate_inodes() in the spin lock context

In reply to Miao Xie

btrfs_invalidate_inodes() may sleep, so we should not invoke it in the spin lock context. Fix it.

---
 fs/btrfs/disk-io.c | 6 ++++++
 1 file changed, 6 insertions(+)
fs/btrfs/disk-io.c +6 -0
index 642c861..724a0da 100644
@@ -3685,8 +3685,11 @@ static void btrfs_destroy_ordered_operations(struct btrfs_transaction *t,
 					 ordered_operations);
 
 		list_del_init(&btrfs_inode->ordered_operations);
+		spin_unlock(&root->fs_info->ordered_extent_lock);
 
 		btrfs_invalidate_inodes(btrfs_inode->root);
+
+		spin_lock(&root->fs_info->ordered_extent_lock);
 	}
 
 	spin_unlock(&root->fs_info->ordered_extent_lock);
@@ -3808,8 +3811,11 @@ static void btrfs_destroy_delalloc_inodes(struct btrfs_root *root)
 		list_del_init(&btrfs_inode->delalloc_inodes);
 		clear_bit(BTRFS_INODE_IN_DELALLOC_LIST,
 			  &btrfs_inode->runtime_flags);
+		spin_unlock(&root->fs_info->delalloc_lock);
 
 		btrfs_invalidate_inodes(btrfs_inode->root);
+
+		spin_lock(&root->fs_info->delalloc_lock);
 	}
 
 	spin_unlock(&root->fs_info->delalloc_lock);
Signature · 1 line
1.8.1.4
author
#20 lore raw

PATCH 8/17 Btrfs: introduce per-subvolume delalloc inode list

In reply to Miao Xie

When we create a snapshot, we need flush all delalloc inodes in the fs, just flushing the inodes in the source tree is OK. So we introduce per-subvolume delalloc inode list.

---
 fs/btrfs/ctree.h       |  22 ++++---
 fs/btrfs/dev-replace.c |   2 +-
 fs/btrfs/disk-io.c     |  49 ++++++++++++---
 fs/btrfs/extent-tree.c |   6 +-
 fs/btrfs/inode.c       | 167 ++++++++++++++++++++++++++++++++++++-------------
 fs/btrfs/relocation.c  |   2 +-
 fs/btrfs/transaction.c |   2 +-
 7 files changed, 183 insertions(+), 67 deletions(-)
fs/btrfs/ctree.h +15 -7
index 958ce6c..067233f 100644
@@ -1449,13 +1449,9 @@ struct btrfs_fs_info {
 	 */
 	struct list_head ordered_extents;
 
-	spinlock_t delalloc_lock;
-	/*
-	 * all of the inodes that have delalloc bytes.  It is possible for
-	 * this list to be empty even when there is still dirty data=ordered
-	 * extents waiting to finish IO.
-	 */
-	struct list_head delalloc_inodes;
+	spinlock_t delalloc_root_lock;
+	/* all fs/file tree roots that have delalloc inodes. */
+	struct list_head delalloc_roots;
 
 	/*
 	 * there is a pool of worker threads for checksumming during writes
@@ -1740,6 +1736,16 @@ struct btrfs_root {
 
 	spinlock_t root_item_lock;
 	atomic_t refs;
+
+	spinlock_t delalloc_lock;
+	/*
+	 * all of the inodes that have delalloc bytes.  It is possible for
+	 * this list to be empty even when there is still dirty data=ordered
+	 * extents waiting to finish IO.
+	 */
+	struct list_head delalloc_inodes;
+	struct list_head delalloc_root;
+	u64 nr_delalloc_inodes;
 };
 
 struct btrfs_ioctl_defrag_range_args {
@@ -3543,6 +3549,8 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			       u32 min_type);
 
 int btrfs_start_delalloc_inodes(struct btrfs_root *root, int delay_iput);
+int btrfs_start_all_delalloc_inodes(struct btrfs_fs_info *fs_info,
+				    int delay_iput);
 int btrfs_set_extent_delalloc(struct inode *inode, u64 start, u64 end,
 			      struct extent_state **cached_state);
 int btrfs_create_subvol_root(struct btrfs_trans_handle *trans,
fs/btrfs/dev-replace.c +1 -1
index 7ba7b39..aca77ad 100644
@@ -465,7 +465,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
 	 * flush all outstanding I/O and inode extent mappings before the
 	 * copy operation is declared as being finished
 	 */
-	ret = btrfs_start_delalloc_inodes(root, 0);
+	ret = btrfs_start_all_delalloc_inodes(root->fs_info, 0);
 	if (ret) {
 		mutex_unlock(&dev_replace->lock_finishing_cancel_unmount);
 		return ret;
fs/btrfs/disk-io.c +39 -10

95 lines not on this page · raw message

fs/btrfs/extent-tree.c +3 -3
index 08f9862..455117a 100644
@@ -3897,7 +3897,7 @@ static void btrfs_writeback_inodes_sb_nr(struct btrfs_root *root,
 		 * the filesystem is readonly(all dirty pages are written to
 		 * the disk).
 		 */
-		btrfs_start_delalloc_inodes(root, 0);
+		btrfs_start_all_delalloc_inodes(root->fs_info, 0);
 		if (!current->journal_info)
 			btrfs_wait_ordered_extents(root, 0);
 	}
@@ -5026,14 +5026,14 @@ static int update_block_group(struct btrfs_root *root,
 	int factor;
 
 	/* block accounting for super block */
-	spin_lock(&info->delalloc_lock);
+	spin_lock(&info->delalloc_root_lock);
 	old_val = btrfs_super_bytes_used(info->super_copy);
 	if (alloc)
 		old_val += num_bytes;
 	else
 		old_val -= num_bytes;
 	btrfs_set_super_bytes_used(info->super_copy, old_val);
-	spin_unlock(&info->delalloc_lock);
+	spin_unlock(&info->delalloc_root_lock);
 
 	while (total) {
 		cache = btrfs_lookup_block_group(info, bytenr);
fs/btrfs/inode.c +123 -44

230 lines not on this page · raw message

fs/btrfs/relocation.c +1 -1
index baaa66b..247fc71 100644
@@ -4153,7 +4153,7 @@ int btrfs_relocate_block_group(struct btrfs_root *extent_root, u64 group_start)
 	       (unsigned long long)rc->block_group->key.objectid,
 	       (unsigned long long)rc->block_group->flags);
 
-	ret = btrfs_start_delalloc_inodes(fs_info->tree_root, 0);
+	ret = btrfs_start_all_delalloc_inodes(fs_info, 0);
 	if (ret < 0) {
 		err = ret;
 		goto out;
fs/btrfs/transaction.c +1 -1
index f157752..4b63111 100644
@@ -1502,7 +1502,7 @@ static int btrfs_flush_all_pending_stuffs(struct btrfs_trans_handle *trans,
 	}
 
 	if (flush_on_commit || snap_pending) {
-		ret = btrfs_start_delalloc_inodes(root, 1);
+		ret = btrfs_start_all_delalloc_inodes(root->fs_info, 1);
 		if (ret)
 			return ret;
 		btrfs_wait_ordered_extents(root, 1);
Signature · 1 line
1.8.1.4
author
#21 lore raw

PATCH 9/17 Btrfs: introduce per-subvolume ordered extent list

In reply to Miao Xie

The reason we introduce per-subvolume ordered extent list is the same as the per-subvolume delalloc inode list.

---
 fs/btrfs/ctree.h        |  25 ++++++++---
 fs/btrfs/dev-replace.c  |   4 +-
 fs/btrfs/disk-io.c      |  45 +++++++++++++++-----
 fs/btrfs/extent-tree.c  |   6 +--
 fs/btrfs/inode.c        |   4 +-
 fs/btrfs/ordered-data.c | 109 +++++++++++++++++++++++++++++++++---------------
 fs/btrfs/ordered-data.h |   2 +
 fs/btrfs/relocation.c   |   2 +-
 fs/btrfs/super.c        |   2 +-
 fs/btrfs/transaction.c  |   2 +-
 10 files changed, 143 insertions(+), 58 deletions(-)
fs/btrfs/ctree.h +20 -5
index 067233f..a7e71ff 100644
@@ -1437,17 +1437,18 @@ struct btrfs_fs_info {
 	atomic_t open_ioctl_trans;
 
 	/*
-	 * this is used by the balancing code to wait for all the pending
-	 * ordered extents
+	 * this is used to protect the following list -- ordered_roots.
 	 */
-	spinlock_t ordered_extent_lock;
+	spinlock_t ordered_root_lock;
 
 	/*
-	 * all of the data=ordered extents pending writeback
+	 * all fs/file tree roots in which there are data=ordered extents
+	 * pending writeback are added into this list.
+	 *
 	 * these can span multiple transactions and basically include
 	 * every dirty data page that isn't from nodatacow
 	 */
-	struct list_head ordered_extents;
+	struct list_head ordered_roots;
 
 	spinlock_t delalloc_root_lock;
 	/* all fs/file tree roots that have delalloc inodes. */
@@ -1746,6 +1747,20 @@ struct btrfs_root {
 	struct list_head delalloc_inodes;
 	struct list_head delalloc_root;
 	u64 nr_delalloc_inodes;
+	/*
+	 * this is used by the balancing code to wait for all the pending
+	 * ordered extents
+	 */
+	spinlock_t ordered_extent_lock;
+
+	/*
+	 * all of the data=ordered extents pending writeback
+	 * these can span multiple transactions and basically include
+	 * every dirty data page that isn't from nodatacow
+	 */
+	struct list_head ordered_extents;
+	struct list_head ordered_root;
+	u64 nr_ordered_extents;
 };
 
 struct btrfs_ioctl_defrag_range_args {
fs/btrfs/dev-replace.c +2 -2
index aca77ad..4254da8 100644
@@ -395,7 +395,7 @@ int btrfs_dev_replace_start(struct btrfs_root *root,
 	args->result = BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR;
 	btrfs_dev_replace_unlock(dev_replace);
 
-	btrfs_wait_ordered_extents(root, 0);
+	btrfs_wait_all_ordered_extents(root->fs_info, 0);
 
 	/* force writing the updated state information to disk */
 	trans = btrfs_start_transaction(root, 0);
@@ -470,7 +470,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
 		mutex_unlock(&dev_replace->lock_finishing_cancel_unmount);
 		return ret;
 	}
-	btrfs_wait_ordered_extents(root, 0);
+	btrfs_wait_all_ordered_extents(root->fs_info, 0);
 
 	trans = btrfs_start_transaction(root, 0);
 	if (IS_ERR(trans)) {
fs/btrfs/disk-io.c +35 -10

103 lines not on this page · raw message

fs/btrfs/extent-tree.c +3 -3
index 455117a..3149616 100644
@@ -3899,7 +3899,7 @@ static void btrfs_writeback_inodes_sb_nr(struct btrfs_root *root,
 		 */
 		btrfs_start_all_delalloc_inodes(root->fs_info, 0);
 		if (!current->journal_info)
-			btrfs_wait_ordered_extents(root, 0);
+			btrfs_wait_all_ordered_extents(root->fs_info, 0);
 	}
 }
 
@@ -3929,7 +3929,7 @@ static void shrink_delalloc(struct btrfs_root *root, u64 to_reclaim, u64 orig,
 	if (delalloc_bytes == 0) {
 		if (trans)
 			return;
-		btrfs_wait_ordered_extents(root, 0);
+		btrfs_wait_all_ordered_extents(root->fs_info, 0);
 		return;
 	}
 
@@ -3957,7 +3957,7 @@ static void shrink_delalloc(struct btrfs_root *root, u64 to_reclaim, u64 orig,
 
 		loops++;
 		if (wait_ordered && !trans) {
-			btrfs_wait_ordered_extents(root, 0);
+			btrfs_wait_all_ordered_extents(root->fs_info, 0);
 		} else {
 			time_left = schedule_timeout_killable(1);
 			if (time_left)
fs/btrfs/inode.c +2 -2
index a270083..69ca278 100644
@@ -7975,9 +7975,9 @@ void btrfs_destroy_inode(struct inode *inode)
 	 */
 	smp_mb();
 	if (!list_empty(&BTRFS_I(inode)->ordered_operations)) {
-		spin_lock(&root->fs_info->ordered_extent_lock);
+		spin_lock(&root->fs_info->ordered_root_lock);
 		list_del_init(&BTRFS_I(inode)->ordered_operations);
-		spin_unlock(&root->fs_info->ordered_extent_lock);
+		spin_unlock(&root->fs_info->ordered_root_lock);
 	}
 
 	if (test_bit(BTRFS_INODE_HAS_ORPHAN_ITEM,
fs/btrfs/ordered-data.c +76 -33

206 lines not on this page · raw message

fs/btrfs/ordered-data.h +2 -0
index 58b0e3b..d082d43 100644
@@ -204,6 +204,8 @@ void btrfs_add_ordered_operation(struct btrfs_trans_handle *trans,
 				 struct btrfs_root *root,
 				 struct inode *inode);
 void btrfs_wait_ordered_extents(struct btrfs_root *root, int delay_iput);
+void btrfs_wait_all_ordered_extents(struct btrfs_fs_info *fs_info,
+				    int delay_iput);
 void btrfs_get_logged_extents(struct btrfs_root *log, struct inode *inode);
 void btrfs_wait_logged_extents(struct btrfs_root *log, u64 transid);
 void btrfs_free_logged_extents(struct btrfs_root *log, u64 transid);
fs/btrfs/relocation.c +1 -1
index 247fc71..f04e703 100644
@@ -4158,7 +4158,7 @@ int btrfs_relocate_block_group(struct btrfs_root *extent_root, u64 group_start)
 		err = ret;
 		goto out;
 	}
-	btrfs_wait_ordered_extents(fs_info->tree_root, 0);
+	btrfs_wait_all_ordered_extents(fs_info, 0);
 
 	while (1) {
 		mutex_lock(&fs_info->cleaner_mutex);
fs/btrfs/super.c +1 -1
index f0857e0..d36d6e6 100644
@@ -866,7 +866,7 @@ int btrfs_sync_fs(struct super_block *sb, int wait)
 		return 0;
 	}
 
-	btrfs_wait_ordered_extents(root, 1);
+	btrfs_wait_all_ordered_extents(fs_info, 0);
 
 	trans = btrfs_attach_transaction_barrier(root);
 	if (IS_ERR(trans)) {
fs/btrfs/transaction.c +1 -1
index 4b63111..2b17213 100644
@@ -1505,7 +1505,7 @@ static int btrfs_flush_all_pending_stuffs(struct btrfs_trans_handle *trans,
 		ret = btrfs_start_all_delalloc_inodes(root->fs_info, 1);
 		if (ret)
 			return ret;
-		btrfs_wait_ordered_extents(root, 1);
+		btrfs_wait_all_ordered_extents(root->fs_info, 1);
 	}
 
 	ret = btrfs_run_delayed_items(trans, root);
Signature · 1 line
1.8.1.4
author
#22 lore raw

PATCH 10/17 Btrfs: just flush the delalloc inodes in the source tree before snapshot creation

In reply to Miao Xie

Before applying this patch, we need flush all the delalloc inodes in the fs when we want to create a snapshot, it wastes time, and make the transaction commit be blocked for a long time. It means some other user operation would also be blocked for a long time.

This patch improves this problem, we just flush the delalloc inodes that in the source trees before snapshot creation, so the transaction commit will complete quickly.

---
 fs/btrfs/ioctl.c       |  6 ++++++
 fs/btrfs/transaction.c | 10 +---------
 2 files changed, 7 insertions(+), 9 deletions(-)
fs/btrfs/ioctl.c +6 -0
index 0de4a2f..2677dcc 100644
@@ -555,6 +555,12 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir,
 	if (!root->ref_cows)
 		return -EINVAL;
 
+	ret = btrfs_start_delalloc_inodes(root, 0);
+	if (ret)
+		return ret;
+
+	btrfs_wait_ordered_extents(root, 0);
+
 	pending_snapshot = kzalloc(sizeof(*pending_snapshot), GFP_NOFS);
 	if (!pending_snapshot)
 		return -ENOMEM;
fs/btrfs/transaction.c +1 -9
index 2b17213..bc22be9 100644
@@ -1491,17 +1491,9 @@ static int btrfs_flush_all_pending_stuffs(struct btrfs_trans_handle *trans,
 					  struct btrfs_root *root)
 {
 	int flush_on_commit = btrfs_test_opt(root, FLUSHONCOMMIT);
-	int snap_pending = 0;
 	int ret;
 
-	if (!flush_on_commit) {
-		spin_lock(&root->fs_info->trans_lock);
-		if (!list_empty(&trans->transaction->pending_snapshots))
-			snap_pending = 1;
-		spin_unlock(&root->fs_info->trans_lock);
-	}
-
-	if (flush_on_commit || snap_pending) {
+	if (flush_on_commit) {
 		ret = btrfs_start_all_delalloc_inodes(root->fs_info, 1);
 		if (ret)
 			return ret;
Signature · 1 line
1.8.1.4
#23 lore raw

Re: [PATCH 10/17] Btrfs: just flush the delalloc inodes in the source tree before snapshot creation

Quoted 20 lines · Miao Xie

Before applying this patch, we need flush all the delalloc inodes in the fs when we want to create a snapshot, it wastes time, and make the transaction commit be blocked for a long time. It means some other user operation would also be blocked for a long time.

This patch improves this problem, we just flush the delalloc inodes that in the source trees before snapshot creation, so the transaction commit will complete quickly.

---
 fs/btrfs/ioctl.c       |  6 ++++++
 fs/btrfs/transaction.c | 10 +---------
 2 files changed, 7 insertions(+), 9 deletions(-)
fs/btrfs/ioctl.c +0 -0
index 0de4a2f..2677dcc 100644

fs/btrfs/ioctl.c static int create_snapshot(struct btrfs_root *root, struct inode *dir, · line 563

Quoted 9 lines
fs/btrfs/ioctl.c +6 -0
@@ -555,6 +555,12 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir,
 	if (!root->ref_cows)
 		return -EINVAL;
 
+	ret = btrfs_start_delalloc_inodes(root, 0);
+	if (ret)
+		return ret;
+
+	btrfs_wait_ordered_extents(root, 0);
+

Does this look too radical? Does this snapshot creation ioctl block all writes on its src root?
No, we can only be sure that there is no ordered extents being added until setting trans_no_join, and then it's safe to create pending snapshots.

thanks,
liubo

Quoted 33 lines
fs/btrfs/ioctl.c +0 -0
 	pending_snapshot = kzalloc(sizeof(*pending_snapshot), GFP_NOFS);
 	if (!pending_snapshot)
 		return -ENOMEM;
fs/btrfs/transaction.c +1 -9
index 2b17213..bc22be9 100644
@@ -1491,17 +1491,9 @@ static int btrfs_flush_all_pending_stuffs(struct btrfs_trans_handle *trans,
 					  struct btrfs_root *root)
 {
 	int flush_on_commit = btrfs_test_opt(root, FLUSHONCOMMIT);
-	int snap_pending = 0;
 	int ret;
 
-	if (!flush_on_commit) {
-		spin_lock(&root->fs_info->trans_lock);
-		if (!list_empty(&trans->transaction->pending_snapshots))
-			snap_pending = 1;
-		spin_unlock(&root->fs_info->trans_lock);
-	}
-
-	if (flush_on_commit || snap_pending) {
+	if (flush_on_commit) {
 		ret = btrfs_start_all_delalloc_inodes(root->fs_info, 1);
 		if (ret)
 			return ret;
Signature · 6 lines
1.8.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#24 lore raw

Re: [PATCH 10/17] Btrfs: just flush the delalloc inodes in the source tree before snapshot creation

Quoted 24 lines · Liu Bo
Quoted 23 lines · Miao Xie

Before applying this patch, we need flush all the delalloc inodes in the fs when we want to create a snapshot, it wastes time, and make the transaction commit be blocked for a long time. It means some other user operation would also be blocked for a long time.

This patch improves this problem, we just flush the delalloc inodes that in the source trees before snapshot creation, so the transaction commit will complete quickly.

---
 fs/btrfs/ioctl.c       |  6 ++++++
 fs/btrfs/transaction.c | 10 +---------
 2 files changed, 7 insertions(+), 9 deletions(-)
fs/btrfs/ioctl.c +0 -0
index 0de4a2f..2677dcc 100644
@@ -555,6 +555,12 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir,
 	if (!root->ref_cows)
 		return -EINVAL;
 
Quoted 9 lines

fs/btrfs/ioctl.c static int create_snapshot(struct btrfs_root *root, struct inode *dir, · line 563

Quoted 6 lines
fs/btrfs/ioctl.c +6 -0
+	ret = btrfs_start_delalloc_inodes(root, 0);
+	if (ret)
+		return ret;
+
+	btrfs_wait_ordered_extents(root, 0);
+

Does this look too radical? Does this snapshot creation ioctl block all writes on its src root?

I don't think it is radical, and I think flushing delalloc inodes during the transaction commit is stupid, especially flushing all the inodes including the roots which are not going to be snapshoted. Because it will block the operations of the users (such as mkdir, rmdir, create and so on) for a long time if there are lots of dirty pages.

And The snapshot creation now doesn't block the writes of the source root at all, there is no appreciable difference between this way and the background flusher.

Quoted 2 lines

No, we can only be sure that there is no ordered extents being added until setting trans_no_join, and then it's safe to create pending snapshots.

Actually, we can not avoid that the new ordered extents are added before trans_no_join is set. But for the users, the 1st case below must be handled correctly, but the 2nd one can be ignored because we can see the write of the 2nd case as the one that happens after the snapshot creation.

1st case:
Task
write data into a file
make a snapshot

2nd case:
Task0 Task1
make a snapshot
flush delalloc inodes
write data into a file
commit transaction
create_pending_snapshot

Thanks
Miao

Quoted 38 lines

thanks,
liubo

Quoted 33 lines
fs/btrfs/ioctl.c +0 -0
 	pending_snapshot = kzalloc(sizeof(*pending_snapshot), GFP_NOFS);
 	if (!pending_snapshot)
 		return -ENOMEM;
fs/btrfs/transaction.c +1 -9
index 2b17213..bc22be9 100644
@@ -1491,17 +1491,9 @@ static int btrfs_flush_all_pending_stuffs(struct btrfs_trans_handle *trans,
 					  struct btrfs_root *root)
 {
 	int flush_on_commit = btrfs_test_opt(root, FLUSHONCOMMIT);
-	int snap_pending = 0;
 	int ret;
 
-	if (!flush_on_commit) {
-		spin_lock(&root->fs_info->trans_lock);
-		if (!list_empty(&trans->transaction->pending_snapshots))
-			snap_pending = 1;
-		spin_unlock(&root->fs_info->trans_lock);
-	}
-
-	if (flush_on_commit || snap_pending) {
+	if (flush_on_commit) {
 		ret = btrfs_start_all_delalloc_inodes(root->fs_info, 1);
 		if (ret)
 			return ret;
Signature · 6 lines
1.8.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#25 lore raw

PATCH 11/17 Btrfs: cleanup unnecessary assignment when cleaning up all the residual transaction

In reply to Miao Xie

When we umount a fs with serious errors, we will invoke btrfs_cleanup_transactions() to clean up the residual transaction. At this time, It is impossible to start a new transaction, so we needn't assign trans_no_join to 1, and also needn't clear running transaction every time we destroy a residual transaction.

---
 fs/btrfs/disk-io.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)
fs/btrfs/disk-io.c +1 -8
index 44d5a86..6bb3f3d 100644
@@ -3992,7 +3992,7 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
 
 	spin_lock(&root->fs_info->trans_lock);
 	list_splice_init(&root->fs_info->trans_list, &list);
-	root->fs_info->trans_no_join = 1;
+	root->fs_info->running_transaction = NULL;
 	spin_unlock(&root->fs_info->trans_lock);
 
 	while (!list_empty(&list)) {
@@ -4028,10 +4028,6 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
 
 		btrfs_destroy_all_delalloc_inodes(root->fs_info);
 
-		spin_lock(&root->fs_info->trans_lock);
-		root->fs_info->running_transaction = NULL;
-		spin_unlock(&root->fs_info->trans_lock);
-
 		btrfs_destroy_marked_extents(root, &t->dirty_pages,
 					     EXTENT_DIRTY);
 
@@ -4044,9 +4040,6 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
 		kmem_cache_free(btrfs_transaction_cachep, t);
 	}
 
-	spin_lock(&root->fs_info->trans_lock);
-	root->fs_info->trans_no_join = 0;
-	spin_unlock(&root->fs_info->trans_lock);
 	mutex_unlock(&root->fs_info->transaction_kthread_mutex);
 
 	return 0;
Signature · 1 line
1.8.1.4
author
#26 lore raw

PATCH 12/17 Btrfs: remove the code for the impossible case in cleanup_transaction()

In reply to Miao Xie

If the transaction is removed from the transaction list, it means the transaction has been committed successfully. So it is impossible to call cleanup_transaction(), otherwise there is something wrong with the code logic. Thus, we use BUG_ON() instead of the original handle.

---
 fs/btrfs/transaction.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
fs/btrfs/transaction.c +6 -5
index bc22be9..cf8706c 100644
@@ -1450,11 +1450,12 @@ static void cleanup_transaction(struct btrfs_trans_handle *trans,
 
 	spin_lock(&root->fs_info->trans_lock);
 
-	if (list_empty(&cur_trans->list)) {
-		spin_unlock(&root->fs_info->trans_lock);
-		btrfs_end_transaction(trans, root);
-		return;
-	}
+	/*
+	 * If the transaction is removed from the list, it means this
+	 * transaction has been committed successfully, so it is impossible
+	 * to call the cleanup function.
+	 */
+	BUG_ON(list_empty(&cur_trans->list));
 
 	list_del_init(&cur_trans->list);
 	if (cur_trans == root->fs_info->running_transaction) {
Signature · 1 line
1.8.1.4
author
#27 lore raw

PATCH 13/17 Btrfs: don't wait for all the writers circularly during the transaction commit

In reply to Miao Xie

btrfs_commit_transaction has the following loop before we commit the transaction.

do {
    // attempt to do some useful stuff and/or sleep
} while (atomic_read(&cur_trans->num_writers) > 1 ||
	 (should_grow && cur_trans->num_joined != joined));

This is used to prevent from the TRANS_START to get in the way of a committing transaction. But it does not prevent from TRANS_JOIN, that is we would do this loop for a long time if some writers JOIN the current transaction endlessly.

Because we need join the current transaction to do some useful stuff, we can not block TRANS_JOIN here. So we introduce a external writer counter, which is used to count the TRANS_USERSPACE/TRANS_START writers. If the external writer counter is zero, we can break the above loop.

In order to make the code more clear, we don't use enum variant to define the type of the transaction handle, use bitmask instead.

---
 fs/btrfs/transaction.c | 55 ++++++++++++++++++++++++++++++++++++++------------
 fs/btrfs/transaction.h | 31 ++++++++++++++++++++--------
 2 files changed, 65 insertions(+), 21 deletions(-)
fs/btrfs/transaction.c +42 -13
index cf8706c..fd319b2 100644
@@ -51,17 +51,41 @@ static noinline void switch_commit_root(struct btrfs_root *root)
 }
 
 static inline int can_join_transaction(struct btrfs_transaction *trans,
-				       int type)
+				       unsigned int type)
 {
 	return !(trans->in_commit &&
-		 type != TRANS_JOIN &&
-		 type != TRANS_JOIN_NOLOCK);
+		 (type & TRANS_EXTWRITERS));
+}
+
+static inline void extwriter_counter_inc(struct btrfs_transaction *trans,
+					 unsigned int type)
+{
+	if (type & TRANS_EXTWRITERS)
+		atomic_inc(&trans->num_extwriters);
+}
+
+static inline void extwriter_counter_dec(struct btrfs_transaction *trans,
+					 unsigned int type)
+{
+	if (type & TRANS_EXTWRITERS)
+		atomic_dec(&trans->num_extwriters);
+}
+
+static inline void extwriter_counter_init(struct btrfs_transaction *trans,
+					  unsigned int type)
+{
+	atomic_set(&trans->num_extwriters, ((type & TRANS_EXTWRITERS) ? 1 : 0));
+}
+
+static inline int extwriter_counter_read(struct btrfs_transaction *trans)
+{
+	return atomic_read(&trans->num_extwriters);
 }
 
 /*
  * either allocate a new transaction or hop into the existing one
  */
-static noinline int join_transaction(struct btrfs_root *root, int type)
+static noinline int join_transaction(struct btrfs_root *root, unsigned int type)
 {
 	struct btrfs_transaction *cur_trans;
 	struct btrfs_fs_info *fs_info = root->fs_info;
@@ -99,6 +123,7 @@ loop:
 		}
 		atomic_inc(&cur_trans->use_count);
 		atomic_inc(&cur_trans->num_writers);
+		extwriter_counter_inc(cur_trans, type);
 		cur_trans->num_joined++;
 		spin_unlock(&fs_info->trans_lock);
 		return 0;
@@ -131,6 +156,7 @@ loop:
 	}
 
 	atomic_set(&cur_trans->num_writers, 1);
+	extwriter_counter_init(cur_trans, type);
 	cur_trans->num_joined = 0;
 	init_waitqueue_head(&cur_trans->writer_wait);
 	init_waitqueue_head(&cur_trans->commit_wait);
@@ -307,7 +333,7 @@ static int may_wait_transaction(struct btrfs_root *root, int type)
 }
 
 static struct btrfs_trans_handle *
-start_transaction(struct btrfs_root *root, u64 num_items, int type,
+start_transaction(struct btrfs_root *root, u64 num_items, unsigned int type,
 		  enum btrfs_reserve_flush_enum flush)
 {
 	struct btrfs_trans_handle *h;
@@ -320,7 +346,7 @@ start_transaction(struct btrfs_root *root, u64 num_items, int type,
 		return ERR_PTR(-EROFS);
 
 	if (current->journal_info) {
-		WARN_ON(type != TRANS_JOIN && type != TRANS_JOIN_NOLOCK);
+		WARN_ON(type & TRANS_EXTWRITERS);
 		h = current->journal_info;
 		h->use_count++;
 		WARN_ON(h->use_count > 2);
@@ -366,7 +392,7 @@ again:
 	 * If we are ATTACH, it means we just want to catch the current
 	 * transaction and commit it, so we needn't do sb_start_intwrite(). 
 	 */
-	if (type < TRANS_JOIN_NOLOCK)
+	if (type & __TRANS_FREEZABLE)
 		sb_start_intwrite(root->fs_info->sb);
 
 	if (may_wait_transaction(root, type))
@@ -429,7 +455,7 @@ got_it:
 	return h;
 
 join_fail:
-	if (type < TRANS_JOIN_NOLOCK)
+	if (type & __TRANS_FREEZABLE)
 		sb_end_intwrite(root->fs_info->sb);
 	kmem_cache_free(btrfs_trans_handle_cachep, h);
 alloc_fail:
@@ -677,12 +703,13 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
 		}
 	}
 
-	if (trans->type < TRANS_JOIN_NOLOCK)
+	if (trans->type & __TRANS_FREEZABLE)
 		sb_end_intwrite(root->fs_info->sb);
 
 	WARN_ON(cur_trans != info->running_transaction);
 	WARN_ON(atomic_read(&cur_trans->num_writers) < 1);
 	atomic_dec(&cur_trans->num_writers);
+	extwriter_counter_dec(cur_trans, trans->type);
 
 	smp_mb();
 	if (waitqueue_active(&cur_trans->writer_wait))
@@ -1625,6 +1652,8 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 		spin_unlock(&root->fs_info->trans_lock);
 	}
 
+	extwriter_counter_dec(cur_trans, trans->type);
+
 	if (!btrfs_test_opt(root, SSD) &&
 	    (now < cur_trans->start_time || now - cur_trans->start_time < 1))
 		should_grow = 1;
@@ -1641,13 +1670,13 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 		prepare_to_wait(&cur_trans->writer_wait, &wait,
 				TASK_UNINTERRUPTIBLE);
 
-		if (atomic_read(&cur_trans->num_writers) > 1)
-			schedule_timeout(MAX_SCHEDULE_TIMEOUT);
+		if (extwriter_counter_read(cur_trans) > 0)
+			schedule();
 		else if (should_grow)
 			schedule_timeout(1);
 
 		finish_wait(&cur_trans->writer_wait, &wait);
-	} while (atomic_read(&cur_trans->num_writers) > 1 ||
+	} while (extwriter_counter_read(cur_trans) > 0 ||
 		 (should_grow && cur_trans->num_joined != joined));
 
 	ret = btrfs_flush_all_pending_stuffs(trans, root);
@@ -1831,7 +1860,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 	put_transaction(cur_trans);
 	put_transaction(cur_trans);
 
-	if (trans->type < TRANS_JOIN_NOLOCK)
+	if (trans->type & __TRANS_FREEZABLE)
 		sb_end_intwrite(root->fs_info->sb);
 
 	trace_btrfs_transaction_commit(root);
fs/btrfs/transaction.h +23 -8
index 24c9733..5cc77b0 100644
@@ -25,6 +25,12 @@
 struct btrfs_transaction {
 	u64 transid;
 	/*
+	 * total external writers(USERSPACE/START/ATTACH) in this
+	 * transaction, it must be zero before the transaction is
+	 * being committed
+	 */
+	atomic_t num_extwriters;
+	/*
 	 * total writers in this transaction, it must be zero before the
 	 * transaction can end
 	 */
@@ -48,13 +54,22 @@ struct btrfs_transaction {
 	int aborted;
 };
 
-enum btrfs_trans_type {
-	TRANS_START,
-	TRANS_JOIN,
-	TRANS_USERSPACE,
-	TRANS_JOIN_NOLOCK,
-	TRANS_ATTACH,
-};
+#define __TRANS_FREEZABLE	(1U << 0)
+
+#define __TRANS_USERSPACE	(1U << 8)
+#define __TRANS_START		(1U << 9)
+#define __TRANS_ATTACH		(1U << 10)
+#define __TRANS_JOIN		(1U << 11)
+#define __TRANS_JOIN_NOLOCK	(1U << 12)
+
+#define TRANS_USERSPACE		(__TRANS_USERSPACE | __TRANS_FREEZABLE)
+#define TRANS_START		(__TRANS_START | __TRANS_FREEZABLE)
+#define TRANS_ATTACH		(__TRANS_ATTACH)
+#define TRANS_JOIN		(__TRANS_JOIN | __TRANS_FREEZABLE)
+#define TRANS_JOIN_NOLOCK	(__TRANS_JOIN_NOLOCK)
+
+#define TRANS_EXTWRITERS	(__TRANS_USERSPACE | __TRANS_START |	\
+				 __TRANS_ATTACH)
 
 struct btrfs_trans_handle {
 	u64 transid;
@@ -70,7 +85,7 @@ struct btrfs_trans_handle {
 	short aborted;
 	short adding_csums;
 	bool allocating_chunk;
-	enum btrfs_trans_type type;
+	unsigned int type;
 	/*
 	 * this root is only needed to validate that the root passed to
 	 * start_transaction is the same as the one passed to end_transaction.
Signature · 1 line
1.8.1.4
author
#28 lore raw

PATCH 14/17 Btrfs: don't flush the delalloc inodes in the while loop if flushoncommit is set

In reply to Miao Xie

It is unnecessary to flush the delalloc inodes again and again because we don't care the dirty pages which are introduced after the flush, and they will be flush in the transaction commit.

---
 fs/btrfs/transaction.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)
fs/btrfs/transaction.c +18 -8
index fd319b2..265db57 100644
@@ -1518,16 +1518,8 @@ static void cleanup_transaction(struct btrfs_trans_handle *trans,
 static int btrfs_flush_all_pending_stuffs(struct btrfs_trans_handle *trans,
 					  struct btrfs_root *root)
 {
-	int flush_on_commit = btrfs_test_opt(root, FLUSHONCOMMIT);
 	int ret;
 
-	if (flush_on_commit) {
-		ret = btrfs_start_all_delalloc_inodes(root->fs_info, 1);
-		if (ret)
-			return ret;
-		btrfs_wait_all_ordered_extents(root->fs_info, 1);
-	}
-
 	ret = btrfs_run_delayed_items(trans, root);
 	if (ret)
 		return ret;
@@ -1551,6 +1543,19 @@ static int btrfs_flush_all_pending_stuffs(struct btrfs_trans_handle *trans,
 	return ret;
 }
 
+static inline int btrfs_start_delalloc_flush(struct btrfs_fs_info *fs_info)
+{
+	if (btrfs_test_opt(fs_info->tree_root, FLUSHONCOMMIT))
+		return btrfs_start_all_delalloc_inodes(fs_info, 1);
+	return 0;
+}
+
+static inline void btrfs_wait_delalloc_flush(struct btrfs_fs_info *fs_info)
+{
+	if (btrfs_test_opt(fs_info->tree_root, FLUSHONCOMMIT))
+		btrfs_wait_all_ordered_extents(fs_info, 1);
+}
+
 /*
  * btrfs_transaction state sequence:
  *    in_commit = 0, blocked = 0  (initial)
@@ -1654,6 +1659,10 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 
 	extwriter_counter_dec(cur_trans, trans->type);
 
+	ret = btrfs_start_delalloc_flush(root->fs_info);
+	if (ret)
+		goto cleanup_transaction;
+
 	if (!btrfs_test_opt(root, SSD) &&
 	    (now < cur_trans->start_time || now - cur_trans->start_time < 1))
 		should_grow = 1;
@@ -1683,6 +1692,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 	if (ret)
 		goto cleanup_transaction;
 
+	btrfs_wait_delalloc_flush(root->fs_info);
 	/*
 	 * Ok now we need to make sure to block out any other joins while we
 	 * commit the transaction.  We could have started a join before setting
Signature · 1 line
1.8.1.4
author
#29 lore raw

PATCH 15/17 Btrfs: remove unnecessary varient ->num_joined in btrfs_transaction structure

In reply to Miao Xie

We used ->num_joined track if there were some writers which join the current transaction when the committer was sleeping. If some writers joined the current transaction, we has to continue the while loop to do some necessary stuff, such as flush the ordered operations. But it is unnecessary because we will do it after the while loop.

Besides that, tracking ->num_joined would make the committer drop into the while loop when there are lots of internal writers(TRANS_JOIN).

So we remove ->num_joined and don't track if there are some writers which join the current transaction when the committer is sleeping.

---
 fs/btrfs/transaction.c | 8 +-------
 fs/btrfs/transaction.h | 2 --
 2 files changed, 1 insertion(+), 9 deletions(-)
fs/btrfs/transaction.c +1 -7
index 265db57..75e7b15 100644
@@ -124,7 +124,6 @@ loop:
 		atomic_inc(&cur_trans->use_count);
 		atomic_inc(&cur_trans->num_writers);
 		extwriter_counter_inc(cur_trans, type);
-		cur_trans->num_joined++;
 		spin_unlock(&fs_info->trans_lock);
 		return 0;
 	}
@@ -157,7 +156,6 @@ loop:
 
 	atomic_set(&cur_trans->num_writers, 1);
 	extwriter_counter_init(cur_trans, type);
-	cur_trans->num_joined = 0;
 	init_waitqueue_head(&cur_trans->writer_wait);
 	init_waitqueue_head(&cur_trans->commit_wait);
 	cur_trans->in_commit = 0;
@@ -1566,7 +1564,6 @@ static inline void btrfs_wait_delalloc_flush(struct btrfs_fs_info *fs_info)
 int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 			     struct btrfs_root *root)
 {
-	unsigned long joined = 0;
 	struct btrfs_transaction *cur_trans = trans->transaction;
 	struct btrfs_transaction *prev_trans = NULL;
 	DEFINE_WAIT(wait);
@@ -1668,8 +1665,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 		should_grow = 1;
 
 	do {
-		joined = cur_trans->num_joined;
-
 		WARN_ON(cur_trans != trans->transaction);
 
 		ret = btrfs_flush_all_pending_stuffs(trans, root);
@@ -1685,8 +1680,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 			schedule_timeout(1);
 
 		finish_wait(&cur_trans->writer_wait, &wait);
-	} while (extwriter_counter_read(cur_trans) > 0 ||
-		 (should_grow && cur_trans->num_joined != joined));
+	} while (extwriter_counter_read(cur_trans) > 0);
 
 	ret = btrfs_flush_all_pending_stuffs(trans, root);
 	if (ret)
fs/btrfs/transaction.h +0 -2
index 5cc77b0..0fc45e2 100644
@@ -37,8 +37,6 @@ struct btrfs_transaction {
 	atomic_t num_writers;
 	atomic_t use_count;
 
-	unsigned long num_joined;
-
 	spinlock_t commit_lock;
 	int in_commit;
 	int commit_done;
Signature · 1 line
1.8.1.4
author
#30 lore raw

PATCH 16/17 Btrfs: remove the time check in btrfs_commit_transaction()

In reply to Miao Xie

We checked the commit time to avoid committing the transaction frequently, but it is unnecessary because:
- It made the transaction commit spend more time, and delayed the operation of the external writers(TRANS_START/TRANS_USERSPACE).
- Except the space that we have to commit transaction, such as snapshot creation, btrfs doesn't commit the transaction on its own initiative.

---
 fs/btrfs/transaction.c | 29 ++++++-----------------------
 1 file changed, 6 insertions(+), 23 deletions(-)
fs/btrfs/transaction.c +6 -23
index 75e7b15..5e75ff4 100644
@@ -1566,10 +1566,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 {
 	struct btrfs_transaction *cur_trans = trans->transaction;
 	struct btrfs_transaction *prev_trans = NULL;
-	DEFINE_WAIT(wait);
 	int ret;
-	int should_grow = 0;
-	unsigned long now = get_seconds();
 
 	ret = btrfs_run_ordered_operations(trans, root, 0);
 	if (ret) {
@@ -1660,28 +1657,14 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
 	if (ret)
 		goto cleanup_transaction;
 
-	if (!btrfs_test_opt(root, SSD) &&
-	    (now < cur_trans->start_time || now - cur_trans->start_time < 1))
-		should_grow = 1;
-
-	do {
-		WARN_ON(cur_trans != trans->transaction);
-
-		ret = btrfs_flush_all_pending_stuffs(trans, root);
-		if (ret)
-			goto cleanup_transaction;
-
-		prepare_to_wait(&cur_trans->writer_wait, &wait,
-				TASK_UNINTERRUPTIBLE);
-
-		if (extwriter_counter_read(cur_trans) > 0)
-			schedule();
-		else if (should_grow)
-			schedule_timeout(1);
+	ret = btrfs_flush_all_pending_stuffs(trans, root);
+	if (ret)
+		goto cleanup_transaction;
 
-		finish_wait(&cur_trans->writer_wait, &wait);
-	} while (extwriter_counter_read(cur_trans) > 0);
+	wait_event(cur_trans->writer_wait,
+		   extwriter_counter_read(cur_trans) == 0);
 
+	/* some pending stuffs might be added after the previous flush. */
 	ret = btrfs_flush_all_pending_stuffs(trans, root);
 	if (ret)
 		goto cleanup_transaction;
Signature · 1 line
1.8.1.4
author
#31 lore raw

PATCH 17/17 Btrfs: make the state of the transaction more readable

In reply to Miao Xie

We used 3 variants to track the state of the transaction, it was complex and wasted the memory space. Besides that, it was hard to understand that which types of the transaction handles should be blocked in each transaction state, so the developers often made mistakes.

This patch improved the above problem. In this patch, we define 6 states for the transaction,

  enum btrfs_trans_state {
	TRANS_STATE_RUNNING		= 0,
	TRANS_STATE_BLOCKED		= 1,
	TRANS_STATE_COMMIT_START	= 2,
	TRANS_STATE_COMMIT_DOING	= 3,
	TRANS_STATE_UNBLOCKED		= 4,
	TRANS_STATE_COMPLETED		= 5,
	TRANS_STATE_MAX			= 6,

}
and just use 1 variant to track those state.

In order to make the blocked handle types for each state more clear, we introduce a array:

  unsigned int btrfs_blocked_trans_types[TRANS_STATE_MAX] = {
	[TRANS_STATE_RUNNING]		= 0U,
	[TRANS_STATE_BLOCKED]		= (__TRANS_USERSPACE |
					   __TRANS_START),
	[TRANS_STATE_COMMIT_START]	= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH),
	[TRANS_STATE_COMMIT_DOING]	= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH |
					   __TRANS_JOIN),
	[TRANS_STATE_UNBLOCKED]		= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH |
					   __TRANS_JOIN |
					   __TRANS_JOIN_NOLOCK),
	[TRANS_STATE_COMPLETED]		= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH |
					   __TRANS_JOIN |
					   __TRANS_JOIN_NOLOCK),

}
it is very intuitionistic.

Besides that, because we remove ->in_commit in transaction structure, so the lock ->commit_lock which was used to protect it is unnecessary, remove ->commit_lock.

---
 fs/btrfs/ctree.h       |   1 -
 fs/btrfs/disk-io.c     |  36 ++++++------
 fs/btrfs/transaction.c | 156 ++++++++++++++++++++++++++-----------------------
 fs/btrfs/transaction.h |  16 +++--
 4 files changed, 114 insertions(+), 95 deletions(-)
fs/btrfs/ctree.h +0 -1
index a7e71ff..bf92302 100644
@@ -1496,7 +1496,6 @@ struct btrfs_fs_info {
 	int closing;
 	int log_root_recovering;
 	int enospc_unlink;
-	int trans_no_join;
 
 	u64 total_pinned;
 
fs/btrfs/disk-io.c +18 -18
index 6bb3f3d..530e3c0 100644
@@ -1747,7 +1747,7 @@ static int transaction_kthread(void *arg)
 		}
 
 		now = get_seconds();
-		if (!cur->blocked &&
+		if (cur->state < TRANS_STATE_BLOCKED &&
 		    (now < cur->start_time || now - cur->start_time < 30)) {
 			spin_unlock(&root->fs_info->trans_lock);
 			delay = HZ * 5;
@@ -2183,7 +2183,6 @@ int open_ctree(struct super_block *sb,
 	fs_info->max_inline = 8192 * 1024;
 	fs_info->metadata_ratio = 0;
 	fs_info->defrag_inodes = RB_ROOT;
-	fs_info->trans_no_join = 0;
 	fs_info->free_chunk_space = 0;
 	fs_info->tree_mod_log = RB_ROOT;
 
@@ -3956,19 +3955,14 @@ void btrfs_cleanup_one_transaction(struct btrfs_transaction *cur_trans,
 	btrfs_block_rsv_release(root, &root->fs_info->trans_block_rsv,
 				cur_trans->dirty_pages.dirty_bytes);
 
-	/* FIXME: cleanup wait for commit */
-	cur_trans->in_commit = 1;
-	cur_trans->blocked = 1;
+	cur_trans->state = TRANS_STATE_COMMIT_START;
 	wake_up(&root->fs_info->transaction_blocked_wait);
 
 	btrfs_evict_pending_snapshots(cur_trans);
 
-	cur_trans->blocked = 0;
+	cur_trans->state = TRANS_STATE_UNBLOCKED;
 	wake_up(&root->fs_info->transaction_wait);
 
-	cur_trans->commit_done = 1;
-	wake_up(&cur_trans->commit_wait);
-
 	btrfs_destroy_delayed_inodes(root);
 	btrfs_assert_delayed_root_empty(root);
 
@@ -3977,6 +3971,9 @@ void btrfs_cleanup_one_transaction(struct btrfs_transaction *cur_trans,
 	btrfs_destroy_pinned_extent(root,
 				    root->fs_info->pinned_extents);
 
+	cur_trans->state =TRANS_STATE_COMPLETED;
+	wake_up(&cur_trans->commit_wait);
+
 	/*
 	memset(cur_trans, 0, sizeof(*cur_trans));
 	kmem_cache_free(btrfs_transaction_cachep, cur_trans);
@@ -4004,25 +4001,23 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
 
 		btrfs_destroy_delayed_refs(t, root);
 
-		/* FIXME: cleanup wait for commit */
-		t->in_commit = 1;
-		t->blocked = 1;
+		/*
+		 *  FIXME: cleanup wait for commit
+		 *  We needn't acquire the lock here, because we are during
+		 *  the umount, there is no other task which will change it.
+		 */
+		t->state = TRANS_STATE_COMMIT_START;
 		smp_mb();
 		if (waitqueue_active(&root->fs_info->transaction_blocked_wait))
 			wake_up(&root->fs_info->transaction_blocked_wait);
 
 		btrfs_evict_pending_snapshots(t);
 
-		t->blocked = 0;
+		t->state = TRANS_STATE_UNBLOCKED;
 		smp_mb();
 		if (waitqueue_active(&root->fs_info->transaction_wait))
 			wake_up(&root->fs_info->transaction_wait);
 
-		t->commit_done = 1;
-		smp_mb();
-		if (waitqueue_active(&t->commit_wait))
-			wake_up(&t->commit_wait);
-
 		btrfs_destroy_delayed_inodes(root);
 		btrfs_assert_delayed_root_empty(root);
 
@@ -4034,6 +4029,11 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
 		btrfs_destroy_pinned_extent(root,
 					    root->fs_info->pinned_extents);
 
+		t->state = TRANS_STATE_COMPLETED;
+		smp_mb();
+		if (waitqueue_active(&t->commit_wait))
+			wake_up(&t->commit_wait);
+
 		atomic_set(&t->use_count, 0);
 		list_del_init(&t->list);
 		memset(t, 0, sizeof(*t));
fs/btrfs/transaction.c +84 -72

352 lines not on this page · raw message

fs/btrfs/transaction.h +12 -4
index 0fc45e2..66d2a6c 100644
@@ -22,6 +22,16 @@
 #include "delayed-ref.h"
 #include "ctree.h"
 
+enum btrfs_trans_state {
+	TRANS_STATE_RUNNING		= 0,
+	TRANS_STATE_BLOCKED		= 1,
+	TRANS_STATE_COMMIT_START	= 2,
+	TRANS_STATE_COMMIT_DOING	= 3,
+	TRANS_STATE_UNBLOCKED		= 4,
+	TRANS_STATE_COMPLETED		= 5,
+	TRANS_STATE_MAX			= 6,
+};
+
 struct btrfs_transaction {
 	u64 transid;
 	/*
@@ -37,10 +47,8 @@ struct btrfs_transaction {
 	atomic_t num_writers;
 	atomic_t use_count;
 
-	spinlock_t commit_lock;
-	int in_commit;
-	int commit_done;
-	int blocked;
+	/* Be protected by fs_info->trans_lock when we want to change it. */
+	enum btrfs_trans_state state;
 	struct list_head list;
 	struct extent_io_tree dirty_pages;
 	unsigned long start_time;
Signature · 1 line
1.8.1.4
author
#32 lore raw

PATCH v2 17/17 Btrfs: make the state of the transaction more readable

We used 3 variants to track the state of the transaction, it was complex and wasted the memory space. Besides that, it was hard to understand that which types of the transaction handles should be blocked in each transaction state, so the developers often made mistakes.

This patch improved the above problem. In this patch, we define 6 states for the transaction,

  enum btrfs_trans_state {
	TRANS_STATE_RUNNING		= 0,
	TRANS_STATE_BLOCKED		= 1,
	TRANS_STATE_COMMIT_START	= 2,
	TRANS_STATE_COMMIT_DOING	= 3,
	TRANS_STATE_UNBLOCKED		= 4,
	TRANS_STATE_COMPLETED		= 5,
	TRANS_STATE_MAX			= 6,

}
and just use 1 variant to track those state.

In order to make the blocked handle types for each state more clear, we introduce a array:

  unsigned int btrfs_blocked_trans_types[TRANS_STATE_MAX] = {
	[TRANS_STATE_RUNNING]		= 0U,
	[TRANS_STATE_BLOCKED]		= (__TRANS_USERSPACE |
					   __TRANS_START),
	[TRANS_STATE_COMMIT_START]	= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH),
	[TRANS_STATE_COMMIT_DOING]	= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH |
					   __TRANS_JOIN),
	[TRANS_STATE_UNBLOCKED]		= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH |
					   __TRANS_JOIN |
					   __TRANS_JOIN_NOLOCK),
	[TRANS_STATE_COMPLETED]		= (__TRANS_USERSPACE |
					   __TRANS_START |
					   __TRANS_ATTACH |
					   __TRANS_JOIN |
					   __TRANS_JOIN_NOLOCK),

}
it is very intuitionistic.

Besides that, because we remove ->in_commit in transaction structure, so the lock ->commit_lock which was used to protect it is unnecessary, remove ->commit_lock.

---
Changelog v1 -> v2:
- remove the misuse of cmpxchg in __btrfs_end_transaction()

---
 fs/btrfs/ctree.h       |   1 -
 fs/btrfs/disk-io.c     |  36 ++++++------
 fs/btrfs/transaction.c | 157 +++++++++++++++++++++++++++----------------------
 fs/btrfs/transaction.h |  16 +++--
 4 files changed, 116 insertions(+), 94 deletions(-)
fs/btrfs/ctree.h +0 -1
index a7e71ff..bf92302 100644
@@ -1496,7 +1496,6 @@ struct btrfs_fs_info {
 	int closing;
 	int log_root_recovering;
 	int enospc_unlink;
-	int trans_no_join;
 
 	u64 total_pinned;
 
fs/btrfs/disk-io.c +18 -18
index 6bb3f3d..530e3c0 100644
@@ -1747,7 +1747,7 @@ static int transaction_kthread(void *arg)
 		}
 
 		now = get_seconds();
-		if (!cur->blocked &&
+		if (cur->state < TRANS_STATE_BLOCKED &&
 		    (now < cur->start_time || now - cur->start_time < 30)) {
 			spin_unlock(&root->fs_info->trans_lock);
 			delay = HZ * 5;
@@ -2183,7 +2183,6 @@ int open_ctree(struct super_block *sb,
 	fs_info->max_inline = 8192 * 1024;
 	fs_info->metadata_ratio = 0;
 	fs_info->defrag_inodes = RB_ROOT;
-	fs_info->trans_no_join = 0;
 	fs_info->free_chunk_space = 0;
 	fs_info->tree_mod_log = RB_ROOT;
 
@@ -3956,19 +3955,14 @@ void btrfs_cleanup_one_transaction(struct btrfs_transaction *cur_trans,
 	btrfs_block_rsv_release(root, &root->fs_info->trans_block_rsv,
 				cur_trans->dirty_pages.dirty_bytes);
 
-	/* FIXME: cleanup wait for commit */
-	cur_trans->in_commit = 1;
-	cur_trans->blocked = 1;
+	cur_trans->state = TRANS_STATE_COMMIT_START;
 	wake_up(&root->fs_info->transaction_blocked_wait);
 
 	btrfs_evict_pending_snapshots(cur_trans);
 
-	cur_trans->blocked = 0;
+	cur_trans->state = TRANS_STATE_UNBLOCKED;
 	wake_up(&root->fs_info->transaction_wait);
 
-	cur_trans->commit_done = 1;
-	wake_up(&cur_trans->commit_wait);
-
 	btrfs_destroy_delayed_inodes(root);
 	btrfs_assert_delayed_root_empty(root);
 
@@ -3977,6 +3971,9 @@ void btrfs_cleanup_one_transaction(struct btrfs_transaction *cur_trans,
 	btrfs_destroy_pinned_extent(root,
 				    root->fs_info->pinned_extents);
 
+	cur_trans->state =TRANS_STATE_COMPLETED;
+	wake_up(&cur_trans->commit_wait);
+
 	/*
 	memset(cur_trans, 0, sizeof(*cur_trans));
 	kmem_cache_free(btrfs_transaction_cachep, cur_trans);
@@ -4004,25 +4001,23 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
 
 		btrfs_destroy_delayed_refs(t, root);
 
-		/* FIXME: cleanup wait for commit */
-		t->in_commit = 1;
-		t->blocked = 1;
+		/*
+		 *  FIXME: cleanup wait for commit
+		 *  We needn't acquire the lock here, because we are during
+		 *  the umount, there is no other task which will change it.
+		 */
+		t->state = TRANS_STATE_COMMIT_START;
 		smp_mb();
 		if (waitqueue_active(&root->fs_info->transaction_blocked_wait))
 			wake_up(&root->fs_info->transaction_blocked_wait);
 
 		btrfs_evict_pending_snapshots(t);
 
-		t->blocked = 0;
+		t->state = TRANS_STATE_UNBLOCKED;
 		smp_mb();
 		if (waitqueue_active(&root->fs_info->transaction_wait))
 			wake_up(&root->fs_info->transaction_wait);
 
-		t->commit_done = 1;
-		smp_mb();
-		if (waitqueue_active(&t->commit_wait))
-			wake_up(&t->commit_wait);
-
 		btrfs_destroy_delayed_inodes(root);
 		btrfs_assert_delayed_root_empty(root);
 
@@ -4034,6 +4029,11 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
 		btrfs_destroy_pinned_extent(root,
 					    root->fs_info->pinned_extents);
 
+		t->state = TRANS_STATE_COMPLETED;
+		smp_mb();
+		if (waitqueue_active(&t->commit_wait))
+			wake_up(&t->commit_wait);
+
 		atomic_set(&t->use_count, 0);
 		list_del_init(&t->list);
 		memset(t, 0, sizeof(*t));
fs/btrfs/transaction.c +86 -71

354 lines not on this page · raw message

fs/btrfs/transaction.h +12 -4
index 0fc45e2..66d2a6c 100644
@@ -22,6 +22,16 @@
 #include "delayed-ref.h"
 #include "ctree.h"
 
+enum btrfs_trans_state {
+	TRANS_STATE_RUNNING		= 0,
+	TRANS_STATE_BLOCKED		= 1,
+	TRANS_STATE_COMMIT_START	= 2,
+	TRANS_STATE_COMMIT_DOING	= 3,
+	TRANS_STATE_UNBLOCKED		= 4,
+	TRANS_STATE_COMPLETED		= 5,
+	TRANS_STATE_MAX			= 6,
+};
+
 struct btrfs_transaction {
 	u64 transid;
 	/*
@@ -37,10 +47,8 @@ struct btrfs_transaction {
 	atomic_t num_writers;
 	atomic_t use_count;
 
-	spinlock_t commit_lock;
-	int in_commit;
-	int commit_done;
-	int blocked;
+	/* Be protected by fs_info->trans_lock when we want to change it. */
+	enum btrfs_trans_state state;
 	struct list_head list;
 	struct extent_io_tree dirty_pages;
 	unsigned long start_time;
Signature · 1 line
1.8.1.4