bcachefs: Increase blacklist range
PATCH 1/1 bcachefs: Increase blacklist range
Now there are 16 journal buffers, 8 is too small to be enough.
--- fs/bcachefs/recovery.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/bcachefs/recovery.c +1 -1
@@ -899,7 +899,7 @@ int bch2_fs_recovery(struct bch_fs *c)* journal sequence numbers: */ if (!c->sb.clean) - journal_seq +=8 ; + journal_seq +=JOURNAL_BUF_NR *4 ;if (blacklist_seq != journal_seq) { ret = bch2_journal_log_msg(c, "blacklisting entries %llu-%llu",
Signature · 1 line
2.48.1
Kent Overstreet
AppliedQuoted 3 lines · Alan Huang
Now there are 16 journal buffers, 8 is too small to be enough.
Signed-off-by Alan Huang <[email protected]>
Applied! Thanks
Quoted 20 lines
--- fs/bcachefs/recovery.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/bcachefs/recovery.c +1
-1@@ -899,7 +899,7 @@ int bch2_fs_recovery(struct bch_fs *c)* journal sequence numbers: */ if (!c->sb.clean) - journal_seq +=8 ; + journal_seq +=JOURNAL_BUF_NR *4 ;if (blacklist_seq != journal_seq) { ret = bch2_journal_log_msg(c, "blacklisting entries %llu-%llu",Signature · 1 line
2.48.1
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Alan" == Alan Huang <[email protected]> writes:
Now there are 16 journal buffers, 8 is too small to be enough.
Signed-off-by Alan Huang <[email protected]>--- fs/bcachefs/recovery.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/bcachefs/recovery.c +0
-0
fs/bcachefs/recovery.c int bch2_fs_recovery(struct bch_fs *c) · line 902
Quoted 5 lines
fs/bcachefs/recovery.c +1
-1@@ -899,7 +899,7 @@ int bch2_fs_recovery(struct bch_fs *c)* journal sequence numbers: */ if (!c->sb.clean) - journal_seq +=8 ; + journal_seq +=JOURNAL_BUF_NR *4 ;
Instead of magic numbers, could you put in a define with an explanation of how you arrived at this number? Just to document the assumptions better?
John
Quoted 23 lines · John Stoffel
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Alan" == Alan Huang <[email protected]> writes:
Now there are 16 journal buffers, 8 is too small to be enough.
Signed-off-by Alan Huang <[email protected]>--- fs/bcachefs/recovery.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/bcachefs/recovery.c +0
-0fs/bcachefs/recovery.c int bch2_fs_recovery(struct bch_fs *c) · line 902
Quoted 5 lines
fs/bcachefs/recovery.c +1
-1@@ -899,7 +899,7 @@ int bch2_fs_recovery(struct bch_fs *c)* journal sequence numbers: */ if (!c->sb.clean) - journal_seq +=8 ; + journal_seq +=JOURNAL_BUF_NR *4 ;Instead of magic numbers, could you put in a define with an explanation of how you arrived at this number? Just to document the assumptions better?
Quoted 1 line
John
The * 4 is a fudge factor.
But actually, I was giving this more thought and I don't think we have the correct number.
The real bound is "number of unflushed journal entries that might have been allocated, and have other items (btree nodes) referring to that sequence number, but which don't hit because beacuse they weren't flushed".
And we don't have an actual bound on that.
Quoted 26 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 23 lines · John Stoffel
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Alan" == Alan Huang <[email protected]> writes:
Now there are 16 journal buffers, 8 is too small to be enough.
Signed-off-by Alan Huang <[email protected]>--- fs/bcachefs/recovery.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/bcachefs/recovery.c +0
-0fs/bcachefs/recovery.c int bch2_fs_recovery(struct bch_fs *c) · line 902
Quoted 5 lines
fs/bcachefs/recovery.c +1
-1@@ -899,7 +899,7 @@ int bch2_fs_recovery(struct bch_fs *c)* journal sequence numbers: */ if (!c->sb.clean) - journal_seq +=8 ; + journal_seq +=JOURNAL_BUF_NR *4 ;Instead of magic numbers, could you put in a define with an explanation of how you arrived at this number? Just to document the assumptions better?
Quoted 3 lines
Quoted 1 line
John
The * 4 is a fudge factor.
Ok.
Quoted 2 lines
But actually, I was giving this more thought and I don't think we have the correct number.
We have a WAG here. :-)
Quoted 6 lines
The real bound is "number of unflushed journal entries that might have been allocated, and have other items (btree nodes) referring to that sequence number, but which don't hit because beacuse they weren't flushed".
And we don't have an actual bound on that.
So what happens if journal_seq overflows? I don't know the code and haven't looked.
Quoted 42 lines · John Stoffel
Quoted 26 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 23 lines · John Stoffel
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Alan" == Alan Huang <[email protected]> writes:
Now there are 16 journal buffers, 8 is too small to be enough.
Signed-off-by Alan Huang <[email protected]>--- fs/bcachefs/recovery.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/bcachefs/recovery.c +0
-0fs/bcachefs/recovery.c int bch2_fs_recovery(struct bch_fs *c) · line 902
Quoted 5 lines
fs/bcachefs/recovery.c +1
-1@@ -899,7 +899,7 @@ int bch2_fs_recovery(struct bch_fs *c)* journal sequence numbers: */ if (!c->sb.clean) - journal_seq +=8 ; + journal_seq +=JOURNAL_BUF_NR *4 ;Instead of magic numbers, could you put in a define with an explanation of how you arrived at this number? Just to document the assumptions better?
Quoted 3 lines
Quoted 1 line
John
The * 4 is a fudge factor.
Ok.
Quoted 2 lines
But actually, I was giving this more thought and I don't think we have the correct number.
We have a WAG here. :-)
Quoted 4 lines
The real bound is "number of unflushed journal entries that might have been allocated, and have other items (btree nodes) referring to that sequence number, but which don't hit because beacuse they weren't flushed".
Quoted 4 lines
Quoted 1 line
And we don't have an actual bound on that.
So what happens if
journal_seqoverflows? I don't know the code and haven't looked.
In the olden days, in the before times, blacklisting insufficient sequence numbers would mean btree node entries could become visible after a crash that should've been ignored, because they were newer than the newest (flush) journal entry.
That can't happen anymore because pointers to btree nodes now indicate how many sectors we've written and completed within that node, and they're updated after every btree node write (log append).
IOW - our mechanism for sequential consistency now is that the b-tree behaves like a pure COW btree.
So I retract what I said before :) Having just checked the journal read path, and the recovery path that creates the blacklist entries, I think we're good.
The only reason we'd need to blacklist more is if we want some kind of a double check on the modern sequential consistency mechanism, and I don't think we need that.
Quoted 61 lines · Kent Overstreet
Quoted 42 lines · John Stoffel
Quoted 26 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 23 lines · John Stoffel
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Alan" == Alan Huang <[email protected]> writes:
Now there are 16 journal buffers, 8 is too small to be enough.
Signed-off-by Alan Huang <[email protected]>---
fs/bcachefs/recovery.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)fs/bcachefs/recovery.c +0
-0fs/bcachefs/recovery.c int bch2_fs_recovery(struct bch_fs *c) · line 902
Quoted 5 lines
fs/bcachefs/recovery.c +1
-1@@ -899,7 +899,7 @@ int bch2_fs_recovery(struct bch_fs *c)* journal sequence numbers: */ if (!c->sb.clean) - journal_seq +=8 ; + journal_seq +=JOURNAL_BUF_NR *4 ;Instead of magic numbers, could you put in a define with an explanation of how you arrived at this number? Just to document the assumptions better?
Quoted 3 lines
Quoted 1 line
John
The * 4 is a fudge factor.
Ok.
Quoted 2 lines
But actually, I was giving this more thought and I don't think we have the correct number.
We have a WAG here. :-)
Quoted 4 lines
The real bound is "number of unflushed journal entries that might have been allocated, and have other items (btree nodes) referring to that sequence number, but which don't hit because beacuse they weren't flushed".
Quoted 4 lines
Quoted 1 line
And we don't have an actual bound on that.
So what happens if
journal_seqoverflows? I don't know the code and haven't looked.In the olden days, in the before times, blacklisting insufficient sequence numbers would mean btree node entries could become visible after a crash that should've been ignored, because they were newer than the newest (flush) journal entry.
That can't happen anymore because pointers to btree nodes now indicate how many sectors we've written and completed within that node, and they're updated after every btree node write (log append).
IOW - our mechanism for sequential consistency now is that the b-tree behaves like a pure COW btree.
Quoted 7 lines
So I retract what I said before :) Having just checked the journal read path, and the recovery path that creates the blacklist entries, I think we're good.
The only reason we'd need to blacklist more is if we want some kind of a double check on the modern sequential consistency mechanism, and I don't think we need that.
What about multi-btree sequential consistency ?
i.e. one btree completes the entire tree update, but the other doesn’t
Quoted 17 lines · Alan Huang
Quoted 16 lines · Kent Overstreet
In the olden days, in the before times, blacklisting insufficient sequence numbers would mean btree node entries could become visible after a crash that should've been ignored, because they were newer than the newest (flush) journal entry.
That can't happen anymore because pointers to btree nodes now indicate how many sectors we've written and completed within that node, and they're updated after every btree node write (log append).
IOW - our mechanism for sequential consistency now is that the b-tree behaves like a pure COW btree.
So I retract what I said before :) Having just checked the journal read path, and the recovery path that creates the blacklist entries, I think we're good.
Quoted 7 lines
Quoted 3 lines
The only reason we'd need to blacklist more is if we want some kind of a double check on the modern sequential consistency mechanism, and I don't think we need that.
What about multi-btree sequential consistency ?
i.e. one btree completes the entire tree update, but the other doesn’t
Btree updates are completed via journal updates, it's all synchronized via the journal