Skip to contentSkip to conversation

btrfs: transaction: Commit transaction more frequently for BPF

PATCH Superseded Qu Wenruo 3 participants 4 messages Aug 6, 2019 Btrfs file system

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH 1/1 btrfs: transaction: Commit transaction more frequently for BPF

Btrfs has btrfs_end_transaction_throttle() which could try to commit transaction when needed.

However under most cases btrfs_end_transaction_throttle() won't really commit transaction, due to the hard timing requirement.

Now introduce a new error injection point, btrfs_need_trans_pressure(), to allow btrfs_should_end_transaction() to return 1 and btrfs_end_transaction_throttle() to fallback to btrfs_commit_transaction().

With such more aggressive transaction commit, we can dig deeper into cases like snapshot drop.
Now each reference drop of btrfs_drop_snapshot() will lead to a transaction commit, allowing dm-logwrites to catch more details, other than one big transaction dropping everything.

---
 fs/btrfs/transaction.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
fs/btrfs/transaction.c +11 -0
index 248d535bb14d..2e758957126e 100644
@@ -10,6 +10,7 @@
 #include <linux/pagemap.h>
 #include <linux/blkdev.h>
 #include <linux/uuid.h>
+#include <linux/error-injection.h>
 #include "ctree.h"
 #include "disk-io.h"
 #include "transaction.h"
@@ -781,10 +782,18 @@ void btrfs_throttle(struct btrfs_fs_info *fs_info)
 	wait_current_trans(fs_info);
 }
 
+static noinline bool btrfs_need_trans_pressure(struct btrfs_trans_handle *trans)
+{
+	return false;
+}
+ALLOW_ERROR_INJECTION(btrfs_need_trans_pressure, TRUE);
+
 static int should_end_transaction(struct btrfs_trans_handle *trans)
 {
 	struct btrfs_fs_info *fs_info = trans->fs_info;
 
+	if (btrfs_need_trans_pressure(trans))
+		return 1;
 	if (btrfs_check_space_for_delayed_refs(fs_info))
 		return 1;
 
@@ -845,6 +854,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
 
 	btrfs_trans_release_chunk_metadata(trans);
 
+	if (throttle && btrfs_need_trans_pressure(trans))
+		return btrfs_commit_transaction(trans);
 	if (lock && READ_ONCE(cur_trans->state) == TRANS_STATE_BLOCKED) {
 		if (throttle)
 			return btrfs_commit_transaction(trans);
Signature · 1 line
2.22.0
#1 lore raw
Quoted 35 lines · Qu Wenruo

Btrfs has btrfs_end_transaction_throttle() which could try to commit transaction when needed.

However under most cases btrfs_end_transaction_throttle() won't really commit transaction, due to the hard timing requirement.

Now introduce a new error injection point, btrfs_need_trans_pressure(), to allow btrfs_should_end_transaction() to return 1 and btrfs_end_transaction_throttle() to fallback to btrfs_commit_transaction().

With such more aggressive transaction commit, we can dig deeper into cases like snapshot drop.
Now each reference drop of btrfs_drop_snapshot() will lead to a transaction commit, allowing dm-logwrites to catch more details, other than one big transaction dropping everything.

---
 fs/btrfs/transaction.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
fs/btrfs/transaction.c +1 -0
index 248d535bb14d..2e758957126e 100644
@@ -10,6 +10,7 @@
 #include <linux/pagemap.h>
 #include <linux/blkdev.h>
 #include <linux/uuid.h>
+#include <linux/error-injection.h>
 #include "ctree.h"
 #include "disk-io.h"
 #include "transaction.h"

fs/btrfs/transaction.c void btrfs_throttle(struct btrfs_fs_info *fs_info) · line 784

Quoted 3 lines
fs/btrfs/transaction.c +0 -0
@@ -781,10 +782,18 @@ void btrfs_throttle(struct btrfs_fs_info *fs_info)
 	wait_current_trans(fs_info);
 }
 

Please put a comment here what's the purpose of the function otherwise the people who like to delete code will find it and you know what will happen next.

Quoted 26 lines
fs/btrfs/transaction.c +10 -0
+static noinline bool btrfs_need_trans_pressure(struct btrfs_trans_handle *trans)
+{
+	return false;
+}
+ALLOW_ERROR_INJECTION(btrfs_need_trans_pressure, TRUE);
+
 static int should_end_transaction(struct btrfs_trans_handle *trans)
 {
 	struct btrfs_fs_info *fs_info = trans->fs_info;
 
+	if (btrfs_need_trans_pressure(trans))
+		return 1;
 	if (btrfs_check_space_for_delayed_refs(fs_info))
 		return 1;
 
@@ -845,6 +854,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
 
 	btrfs_trans_release_chunk_metadata(trans);
 
+	if (throttle && btrfs_need_trans_pressure(trans))
+		return btrfs_commit_transaction(trans);
 	if (lock && READ_ONCE(cur_trans->state) == TRANS_STATE_BLOCKED) {
 		if (throttle)
 			return btrfs_commit_transaction(trans);
Signature · 1 line
2.22.0
#2 lore raw
In reply to Qu Wenruo
Quoted 54 lines · Qu Wenruo

Btrfs has btrfs_end_transaction_throttle() which could try to commit transaction when needed.

However under most cases btrfs_end_transaction_throttle() won't really commit transaction, due to the hard timing requirement.

Now introduce a new error injection point, btrfs_need_trans_pressure(), to allow btrfs_should_end_transaction() to return 1 and btrfs_end_transaction_throttle() to fallback to btrfs_commit_transaction().

With such more aggressive transaction commit, we can dig deeper into cases like snapshot drop.
Now each reference drop of btrfs_drop_snapshot() will lead to a transaction commit, allowing dm-logwrites to catch more details, other than one big transaction dropping everything.

---
 fs/btrfs/transaction.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
fs/btrfs/transaction.c +9 -0
index 248d535bb14d..2e758957126e 100644
@@ -10,6 +10,7 @@
 #include <linux/pagemap.h>
 #include <linux/blkdev.h>
 #include <linux/uuid.h>
+#include <linux/error-injection.h>
 #include "ctree.h"
 #include "disk-io.h"
 #include "transaction.h"
@@ -781,10 +782,18 @@ void btrfs_throttle(struct btrfs_fs_info *fs_info)
 	wait_current_trans(fs_info);
 }
 
+static noinline bool btrfs_need_trans_pressure(struct btrfs_trans_handle *trans)
+{
+	return false;
+}
+ALLOW_ERROR_INJECTION(btrfs_need_trans_pressure, TRUE);
+
 static int should_end_transaction(struct btrfs_trans_handle *trans)
 {
 	struct btrfs_fs_info *fs_info = trans->fs_info;
 
+	if (btrfs_need_trans_pressure(trans))
+		return 1;
 	if (btrfs_check_space_for_delayed_refs(fs_info))
 		return 1;
 

fs/btrfs/transaction.c static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, · line 858

Quoted 5 lines
fs/btrfs/transaction.c +2 -0
@@ -845,6 +854,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
 
 	btrfs_trans_release_chunk_metadata(trans);
 
+	if (throttle && btrfs_need_trans_pressure(trans))
+		return btrfs_commit_transaction(trans);

I'd rather have this gated behind CONFIG_BTRFS_DEBUG.

Quoted 4 lines
fs/btrfs/transaction.c +0 -0
 	if (lock && READ_ONCE(cur_trans->state) == TRANS_STATE_BLOCKED) {
 		if (throttle)
 			return btrfs_commit_transaction(trans);
author
#3 lore raw
Quoted 56 lines · Nikolay Borisov
Quoted 53 lines · Qu Wenruo

Btrfs has btrfs_end_transaction_throttle() which could try to commit transaction when needed.

However under most cases btrfs_end_transaction_throttle() won't really commit transaction, due to the hard timing requirement.

Now introduce a new error injection point, btrfs_need_trans_pressure(), to allow btrfs_should_end_transaction() to return 1 and btrfs_end_transaction_throttle() to fallback to btrfs_commit_transaction().

With such more aggressive transaction commit, we can dig deeper into cases like snapshot drop.
Now each reference drop of btrfs_drop_snapshot() will lead to a transaction commit, allowing dm-logwrites to catch more details, other than one big transaction dropping everything.

---
 fs/btrfs/transaction.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
fs/btrfs/transaction.c +9 -0
index 248d535bb14d..2e758957126e 100644
@@ -10,6 +10,7 @@
 #include <linux/pagemap.h>
 #include <linux/blkdev.h>
 #include <linux/uuid.h>
+#include <linux/error-injection.h>
 #include "ctree.h"
 #include "disk-io.h"
 #include "transaction.h"
@@ -781,10 +782,18 @@ void btrfs_throttle(struct btrfs_fs_info *fs_info)
 	wait_current_trans(fs_info);
 }
 
+static noinline bool btrfs_need_trans_pressure(struct btrfs_trans_handle *trans)
+{
+	return false;
+}
+ALLOW_ERROR_INJECTION(btrfs_need_trans_pressure, TRUE);
+
 static int should_end_transaction(struct btrfs_trans_handle *trans)
 {
 	struct btrfs_fs_info *fs_info = trans->fs_info;
 
+	if (btrfs_need_trans_pressure(trans))
+		return 1;
 	if (btrfs_check_space_for_delayed_refs(fs_info))
 		return 1;
 
Quoted 8 lines

fs/btrfs/transaction.c static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, · line 858

Quoted 6 lines
fs/btrfs/transaction.c +2 -0
@@ -845,6 +854,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
 
 	btrfs_trans_release_chunk_metadata(trans);
 
+	if (throttle && btrfs_need_trans_pressure(trans))
+		return btrfs_commit_transaction(trans);

I'd rather have this gated behind CONFIG_BTRFS_DEBUG.

Then we need both CONFIG_BTRFS_DEBUG and CONFIG_BPF_KPROBE_OVERRIDE to enable this feature.

And for CONFIG_BTRFS_DEBUG not enabled case, that function would not be used at all (if you also gate the one in should_end_transaction()).

Not sure if extra hassle is really worthy for such a simple debug feature.

Thanks,
Qu

Quoted 6 lines
Quoted 6 lines
fs/btrfs/transaction.c +0 -0
 	if (lock && READ_ONCE(cur_trans->state) == TRANS_STATE_BLOCKED) {
 		if (throttle)
 			return btrfs_commit_transaction(trans);