btree write buffer & journal optimizations
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest17 patches 11 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/17 | bcachefs: Kill journal pre-reservationsfs/ fs/ fs/ +8 more | +19 | 0 replies | |
| 2/17 | bcachefs: track_event_change()fs/ fs/ fs/ +5 more | +140 | 0 replies | |
| 3/17 | bcachefs: Journal pins must always have a flush_fnfs/ fs/ fs/ +1 more | +40 | 3 replies | |
| 4/17 | bcachefs: BTREE_INSERT_JOURNAL_REPLAY now "don't init trans->journal_res"fs/ fs/ | +17 | 0 replies | |
| 5/17 | bcachefs: Kill BTREE_UPDATE_PREJOURNALfs/ fs/ fs/ +2 more | +11 | 3 replies | |
| 6/17 | bcachefs: Go rw before journal replayfs/ | +12 | 0 replies | |
| 7/17 | bcachefs: Make journal replay more efficientfs/ fs/ | +56 | 2 replies | |
| 8/17 | bcachefs: Don't flush journal after replayfs/ | +1 | 0 replies | |
| 9/17 | bcachefs: Unwritten journal buffers are always dirtyfs/ fs/ | +2 | 0 replies | |
| 10/17 | bcachefs: journal->buf_lockfs/ fs/ fs/ | +9 | 0 replies | |
| 11/17 | bcachefs: bch2_journal_block_reservations()fs/ fs/ | +31 | 0 replies | |
| 12/17 | bcachefs: Clean up btree write buffer write ref handlingfs/ fs/ fs/ +2 more | +32 | 0 replies | |
| 13/17 | bcachefs: bch2_btree_write_buffer_flush_locked()fs/ fs/ fs/ | +18 | 0 replies | |
| 14/17 | bcachefs: bch2_btree_write_buffer_flush() -> bch2_btree_write_buffer_tryflush()fs/ fs/ fs/ +3 more | +9 | 0 replies | |
| 15/17 | bcachefs: Improve btree write buffer tracepointsfs/ fs/ | +27 | 0 replies | |
| 16/17 | bcachefs: btree write buffer now slurps keys from journalfs/ fs/ fs/ +11 more | +512 | 2 replies | |
| 17/17 | bcachefs: Inline btree write buffer sortfs/ | +48 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [PATCH 00/17] btree write buffer & journal optimizations
Large stack of changes - performance optimizations and improvements to related code.
The first big one, killing journal pre-reservations, is already merged:
besides being a performance improvement it turned out to fix a few tests that were sporadically failing with "journal stuck" - nice. The idea with that patch is that instead of reserving space in the journal ahead of time (for journal reservations that must succeed or risk deadlock), we instead rely on watermarks and tracking which operations are necessary for reclaim - when we're low on space in the journal and flushing things for reclaim, we're able to flush things in order, or track when we're not flushing things in order and allow it to fail
Most of the other patches are prep work leading up to the big btree write buffer rewrite, which changes the btree write buffer to pull keys from the journal, instead of having the tranaction commit path directly add keys to the btree write buffer.
btree write buffer performance is pretty important for multithreaded update workloads - it's an amdahl's law situation, flushing it is single threaded. More performance optimizations there would still be worthwhile, and I've been looking at it because I'm sketching out a rewrite of disk space accounting (which will allow for per snapshot Id accounting) that will lean heavily on the btree write buffer.
Kent Overstreet (17):
bcachefs: Kill journal pre-reservations
bcachefs: track_event_change()
bcachefs: Journal pins must always have a flush_fn
bcachefs: BTREE_INSERT_JOURNAL_REPLAY now "don't init
trans->journal_res"
bcachefs: Kill BTREE_UPDATE_PREJOURNAL
bcachefs: Go rw before journal replay
bcachefs: Make journal replay more efficient
bcachefs: Don't flush journal after replay
bcachefs: Unwritten journal buffers are always dirty
bcachefs: journal->buf_lock
bcachefs: bch2_journal_block_reservations()
bcachefs: Clean up btree write buffer write ref handling
bcachefs: bch2_btree_write_buffer_flush_locked()
bcachefs: bch2_btree_write_buffer_flush() ->
bch2_btree_write_buffer_tryflush()
bcachefs: Improve btree write buffer tracepoints
bcachefs: btree write buffer now slurps keys from journal
bcachefs: Inline btree write buffer sort
fs/bcachefs/alloc_background.c | 2 +- fs/bcachefs/bcachefs.h | 18 +- fs/bcachefs/bcachefs_format.h | 3 +- fs/bcachefs/bkey_methods.h | 2 - fs/bcachefs/btree_iter.c | 2 - fs/bcachefs/btree_key_cache.c | 14 - fs/bcachefs/btree_trans_commit.c | 123 ++---- fs/bcachefs/btree_types.h | 4 - fs/bcachefs/btree_update.c | 23 - fs/bcachefs/btree_update_interior.c | 46 +- fs/bcachefs/btree_update_interior.h | 1 - fs/bcachefs/btree_write_buffer.c | 584 +++++++++++++++++-------- fs/bcachefs/btree_write_buffer.h | 48 +- fs/bcachefs/btree_write_buffer_types.h | 37 +- fs/bcachefs/ec.c | 2 +- fs/bcachefs/errcode.h | 2 - fs/bcachefs/inode.c | 10 +- fs/bcachefs/journal.c | 120 +++-- fs/bcachefs/journal.h | 102 +---- fs/bcachefs/journal_io.c | 57 ++- fs/bcachefs/journal_reclaim.c | 66 +-- fs/bcachefs/journal_reclaim.h | 1 + fs/bcachefs/journal_types.h | 40 +- fs/bcachefs/move.c | 7 +- fs/bcachefs/movinggc.c | 4 +- fs/bcachefs/recovery.c | 106 +++-- fs/bcachefs/super.c | 6 +- fs/bcachefs/trace.h | 42 +- fs/bcachefs/util.c | 140 +++--- fs/bcachefs/util.h | 33 +- 30 files changed, 925 insertions(+), 720 deletions(-)
Signature · 1 line
2.42.0