6.16-rc3 hotfixes series
cover 6.16-rc3 hotfixes series
Busy week...
We now have journal rewind, which one of the users (the one who didn't have backups) who was bit by the subvolume deletion bug was able to use to fully recover his filesystem.
Going to do some work on the discard path so we can have that as a reliable disaster recovery mechanism.
Otherwise, mostly boring repair stuff, some tracepoints for tracking down livelock-ish issues.
Alan Huang (6):
bcachefs: Don't allocate new memory when mempool is exhausted
bcachefs: Fix alloc_req use after free
bcachefs: Add missing EBUG_ON
bcachefs: Delay calculation of trans->journal_u64s
bcachefs: Move bset size check before csum check
bcachefs: Fix pool->alloc NULL pointer dereference
Bharadwaj Raju (1):
bcachefs: don't return fsck_fix for unfixable node errors in
__btree_err
Kent Overstreet (30):
bcachefs: trace_extent_trim_atomic
bcachefs: btree iter tracepoints
bcachefs: Fix bch2_journal_keys_peek_prev_min()
bcachefs: btree_iter: fix updates, journal overlay
bcachefs: better __bch2_snapshot_is_ancestor() assert
bcachefs: pass last_seq into fs_journal_start()
bcachefs: Fix "now allowing incompatible features" message
bcachefs: Fix snapshot_key_missing_inode_snapshot repair
bcachefs: fsck: fix add_inode()
bcachefs: fsck: fix extent past end of inode repair
bcachefs: opts.journal_rewind
bcachefs: Kill unused tracepoints
bcachefs: mark more errors autofix
bcachefs: fsck: Improve check_key_has_inode()
bcachefs: Call bch2_fs_init_rw() early if we'll be going rw
bcachefs: Fix __bch2_inum_to_path() when crossing subvol boundaries
bcachefs: fsck: Print path when we find a subvol loop
bcachefs: fsck: Fix remove_backpointer() for subvol roots
bcachefs: fsck: Fix reattach_inode() for subvol roots
bcachefs: fsck: check_directory_structure runs in reverse order
bcachefs: fsck: additional diagnostics for reattach_inode()
bcachefs: fsck: check_subdir_count logs path
bcachefs: fsck: Fix check_path_loop() + snapshots
bcachefs: Fix bch2_read_bio_to_text()
bcachefs: Fix restart handling in btree_node_scrub_work()
bcachefs: fsck: Fix check_directory_structure when no check_dirents
bcachefs: fsck: fix unhandled restart in topology repair
bcachefs: fsck: Fix oops in key_visible_in_snapshot()
bcachefs: fix spurious error in read_btree_roots()
bcachefs: Fix missing newlines before ero
fs/bcachefs/bcachefs.h | 3 +- fs/bcachefs/btree_gc.c | 8 +- fs/bcachefs/btree_io.c | 74 +++--- fs/bcachefs/btree_iter.c | 169 ++++++++----- fs/bcachefs/btree_journal_iter.c | 64 +++-- fs/bcachefs/btree_journal_iter_types.h | 5 +- fs/bcachefs/btree_trans_commit.c | 18 +- fs/bcachefs/btree_types.h | 1 + fs/bcachefs/btree_update_interior.c | 11 +- fs/bcachefs/btree_update_interior.h | 3 + fs/bcachefs/btree_write_buffer.c | 3 + fs/bcachefs/chardev.c | 29 ++- fs/bcachefs/data_update.c | 1 + fs/bcachefs/errcode.h | 5 - fs/bcachefs/extent_update.c | 13 +- fs/bcachefs/fsck.c | 317 +++++++++++++++++-------- fs/bcachefs/inode.h | 5 + fs/bcachefs/io_read.c | 7 +- fs/bcachefs/journal.c | 18 +- fs/bcachefs/journal.h | 2 +- fs/bcachefs/journal_io.c | 26 +- fs/bcachefs/namei.c | 30 ++- fs/bcachefs/opts.h | 5 + fs/bcachefs/recovery.c | 24 +- fs/bcachefs/recovery_passes.c | 6 +- fs/bcachefs/recovery_passes.h | 9 + fs/bcachefs/sb-errors_format.h | 10 +- fs/bcachefs/snapshot.c | 8 +- fs/bcachefs/super.c | 13 +- fs/bcachefs/super.h | 1 + fs/bcachefs/trace.h | 125 +++------- 31 files changed, 622 insertions(+), 391 deletions(-)
Signature · 1 line
2.50.0
PATCH 1/37 bcachefs: trace_extent_trim_atomic
Add a tracepoint for when we insert only part of an extent, due to too many overwrites.
--- fs/bcachefs/extent_update.c | 13 ++++++++++++- fs/bcachefs/trace.h | 5 +++++ 2 files changed, 17 insertions(+), 1 deletion(-)
fs/bcachefs/extent_update.c +12 -1
@@ -139,6 +139,17 @@ int bch2_extent_trim_atomic(struct btree_trans *trans,if (ret)return ret; -bch2_cut_back (end, k); +/* tracepoint */ + +if (bpos_lt (end, k->k.p)) { +if (trace_extent_trim_atomic_enabled ()) { +CLASS (printbuf, buf)(); +bch2_bpos_to_text (&buf, end); +prt_newline (&buf); +bch2_bkey_val_to_text (&buf, trans->c,bkey_i_to_s_c (k)); +trace_extent_trim_atomic (trans->c, buf.buf); + } +bch2_cut_back (end, k); + }return 0 ; }
fs/bcachefs/trace.h +5 -0
@@ -1490,6 +1490,11 @@ DEFINE_EVENT(fs_str, io_move_evacuate_bucket,TP_ARGS (c, str) ); +DEFINE_EVENT (fs_str, extent_trim_atomic, + TP_PROTO(struct bch_fs *c, const char *str), +TP_ARGS (c, str) +); +#ifdef CONFIG_BCACHEFS_PATH_TRACEPOINTS TRACE_EVENT(update_by_path,
Signature · 1 line
2.50.0
PATCH 2/37 bcachefs: btree iter tracepoints
In reply to Kent OverstreetWe've been seeing some livelock-ish behavior in the index update part of the main write path, and while we've got low level btree path tracepoints, we've been lacking high level btree iterator tracepoints.
--- fs/bcachefs/btree_iter.c | 67 +++++++++++++++++++++++++++++++++++++--- fs/bcachefs/trace.h | 20 ++++++++++++ 2 files changed, 82 insertions(+), 5 deletions(-)
fs/bcachefs/btree_iter.c +62 -5
@@ -2326,6 +2326,20 @@ static struct bkey_s_c __bch2_btree_iter_peek(struct btree_trans *trans, struct }bch2_btree_iter_verify (trans, iter); + +if (trace___btree_iter_peek_enabled ()) { +CLASS (printbuf, buf)(); + +int ret =bkey_err (k); +if (ret) +prt_str (&buf,bch2_err_str (ret)); +else if (k.k) +bch2_bkey_val_to_text (&buf, trans->c, k); +else +prt_str (&buf,"(null)" ); +trace___btree_iter_peek (trans->c, buf.buf); + } +return k; }
@@ -2484,6 +2498,19 @@ struct bkey_s_c bch2_btree_iter_peek_max(struct btree_trans *trans, struct btreebch2_btree_iter_verify_entry_exit (iter); +if (trace_btree_iter_peek_max_enabled ()) { +CLASS (printbuf, buf)(); + +int ret =bkey_err (k); +if (ret) +prt_str (&buf,bch2_err_str (ret)); +else if (k.k) +bch2_bkey_val_to_text (&buf, trans->c, k); +else +prt_str (&buf,"(null)" ); +trace_btree_iter_peek_max (trans->c, buf.buf); + } +return k; end:bch2_btree_iter_set_pos (trans, iter, end);
@@ -2724,6 +2751,19 @@ struct bkey_s_c bch2_btree_iter_peek_prev_min(struct btree_trans *trans, structbch2_btree_iter_verify_entry_exit (iter);bch2_btree_iter_verify (trans, iter); + +if (trace_btree_iter_peek_prev_min_enabled ()) { +CLASS (printbuf, buf)(); + +int ret =bkey_err (k); +if (ret) +prt_str (&buf,bch2_err_str (ret)); +else if (k.k) +bch2_bkey_val_to_text (&buf, trans->c, k); +else +prt_str (&buf,"(null)" ); +trace_btree_iter_peek_prev_min (trans->c, buf.buf); + }return k; end:bch2_btree_iter_set_pos (trans, iter, end);
@@ -2767,8 +2807,10 @@ struct bkey_s_c bch2_btree_iter_peek_slot(struct btree_trans *trans, struct btre/* extents can't span inode numbers: */ if ((iter->flags & BTREE_ITER_is_extents) &&unlikely (iter->pos.offset ==KEY_OFFSET_MAX )) { -if (iter->pos.inode ==KEY_INODE_MAX ) -return bkey_s_c_null; +if (iter->pos.inode ==KEY_INODE_MAX ) { + k = bkey_s_c_null; + goto out2; + }bch2_btree_iter_set_pos (trans, iter,bpos_nosnap_successor (iter->pos)); }
@@ -2785,8 +2827,10 @@ struct bkey_s_c bch2_btree_iter_peek_slot(struct btree_trans *trans, struct btre }struct btree_path *path =btree_iter_path (trans, iter); -if (unlikely (!btree_path_node (path, path->level))) -return bkey_s_c_null; +if (unlikely (!btree_path_node (path, path->level))) { + k = bkey_s_c_null; + goto out2; + }btree_path_set_should_be_locked (trans, path);
@@ -2879,7 +2923,20 @@ struct bkey_s_c bch2_btree_iter_peek_slot(struct btree_trans *trans, struct btrebch2_btree_iter_verify (trans, iter); ret =bch2_btree_iter_verify_ret (trans, iter, k);if (unlikely (ret)) -return bkey_s_c_err (ret); + k =bkey_s_c_err (ret); +out2: +if (trace_btree_iter_peek_slot_enabled ()) { +CLASS (printbuf, buf)(); + +int ret =bkey_err (k); +if (ret) +prt_str (&buf,bch2_err_str (ret)); +else if (k.k) +bch2_bkey_val_to_text (&buf, trans->c, k); +else +prt_str (&buf,"(null)" ); +trace_btree_iter_peek_slot (trans->c, buf.buf); + }return k; }
fs/bcachefs/trace.h +20 -0
@@ -1495,6 +1495,26 @@ DEFINE_EVENT(fs_str, extent_trim_atomic,TP_ARGS (c, str) ); +DEFINE_EVENT (fs_str, btree_iter_peek_slot, + TP_PROTO(struct bch_fs *c, const char *str), +TP_ARGS (c, str) +); + +DEFINE_EVENT (fs_str, __btree_iter_peek, + TP_PROTO(struct bch_fs *c, const char *str), +TP_ARGS (c, str) +); + +DEFINE_EVENT (fs_str, btree_iter_peek_max, + TP_PROTO(struct bch_fs *c, const char *str), +TP_ARGS (c, str) +); + +DEFINE_EVENT (fs_str, btree_iter_peek_prev_min, + TP_PROTO(struct bch_fs *c, const char *str), +TP_ARGS (c, str) +); +#ifdef CONFIG_BCACHEFS_PATH_TRACEPOINTS TRACE_EVENT(update_by_path,
Signature · 1 line
2.50.0
PATCH 3/37 bcachefs: Don't allocate new memory when mempool is exhausted
In reply to Kent OverstreetFrom: Alan Huang <[email protected]>
Allocating new memory when mempool is exhausted is too complicated, just return ENOMEM is fine. memcpy is not needed, since there might be pointers point to the old memory, that's the bug.
--- fs/bcachefs/btree_iter.c | 29 ++++------------------------- 1 file changed, 4 insertions(+), 25 deletions(-)
fs/bcachefs/btree_iter.c +4 -25
@@ -3217,28 +3217,8 @@ void *__bch2_trans_kmalloc(struct btree_trans *trans, size_t size, unsigned long }if (trans->used_mempool) { -if (trans->mem_bytes >= new_bytes) - goto out_change_top; - -/* No more space from mempool item, need malloc new one */ - new_mem =kmalloc (new_bytes,GFP_NOWAIT |__GFP_NOWARN); -if (unlikely (!new_mem)) { -bch2_trans_unlock (trans); - - new_mem =kmalloc (new_bytes,GFP_KERNEL ); -if (!new_mem) -return ERR_PTR (-BCH_ERR_ENOMEM_trans_kmalloc); - - ret =bch2_trans_relock (trans); -if (ret) { -kfree (new_mem); -return ERR_PTR (ret); - } - } -memcpy (new_mem, trans->mem, trans->mem_top); - trans->used_mempool = false; -mempool_free (trans->mem, &c->btree_trans_mem_pool); - goto out_new_mem; +EBUG_ON (trans->mem_bytes >= new_bytes); +return ERR_PTR (-BCH_ERR_ENOMEM_trans_kmalloc); } new_mem =krealloc (trans->mem, new_bytes,GFP_NOWAIT |__GFP_NOWARN);
@@ -3249,7 +3229,6 @@ void *__bch2_trans_kmalloc(struct btree_trans *trans, size_t size, unsigned longif (!new_mem && new_bytes <=BTREE_TRANS_MEM_MAX ) { new_mem =mempool_alloc (&c->btree_trans_mem_pool,GFP_KERNEL ); new_bytes =BTREE_TRANS_MEM_MAX ; -memcpy (new_mem, trans->mem, trans->mem_top); trans->used_mempool = true;kfree (trans->mem); }
@@ -3264,7 +3243,7 @@ void *__bch2_trans_kmalloc(struct btree_trans *trans, size_t size, unsigned longif (ret)return ERR_PTR (ret); } -out_new_mem: + trans->mem = new_mem; trans->mem_bytes = new_bytes;
@@ -3273,7 +3252,7 @@ void *__bch2_trans_kmalloc(struct btree_trans *trans, size_t size, unsigned longreturn ERR_PTR (btree_trans_restart_ip (trans, BCH_ERR_transaction_restart_mem_realloced, _RET_IP_)); } -out_change_top: +bch2_trans_kmalloc_trace (trans, size, ip); p = trans->mem + trans->mem_top;
Signature · 1 line
2.50.0
PATCH 4/37 bcachefs: Fix alloc_req use after free
In reply to Kent OverstreetFrom: Alan Huang <[email protected]>
Now the alloc_req is allocated from the bump allocator, if there is reallocation, the memory of alloc_req would be frees, fix by delaying the reallocation to transaction restart, it has to restart anyway.
--- fs/bcachefs/btree_iter.c | 48 +++++++++++++++++++++++++++------------ fs/bcachefs/btree_types.h | 1 + 2 files changed, 35 insertions(+), 14 deletions(-)
fs/bcachefs/btree_iter.c +34 -14
@@ -3216,25 +3216,32 @@ void *__bch2_trans_kmalloc(struct btree_trans *trans, size_t size, unsigned longmutex_unlock (&s->lock); } -if (trans->used_mempool) { +if (trans->used_mempool || new_bytes >BTREE_TRANS_MEM_MAX ) {EBUG_ON (trans->mem_bytes >= new_bytes);return ERR_PTR (-BCH_ERR_ENOMEM_trans_kmalloc); } - new_mem =krealloc (trans->mem, new_bytes,GFP_NOWAIT |__GFP_NOWARN); +if (old_bytes) { + trans->realloc_bytes_required = new_bytes; +trace_and_count (c, trans_restart_mem_realloced, trans, _RET_IP_, new_bytes); +return ERR_PTR (btree_trans_restart_ip (trans, + BCH_ERR_transaction_restart_mem_realloced, _RET_IP_)); + } + +EBUG_ON (trans->mem); + + new_mem =kmalloc (new_bytes,GFP_NOWAIT |__GFP_NOWARN);if (unlikely (!new_mem)) {bch2_trans_unlock (trans); - new_mem =krealloc (trans->mem, new_bytes,GFP_KERNEL ); + new_mem =kmalloc (new_bytes,GFP_KERNEL );if (!new_mem && new_bytes <=BTREE_TRANS_MEM_MAX ) { new_mem =mempool_alloc (&c->btree_trans_mem_pool,GFP_KERNEL ); new_bytes =BTREE_TRANS_MEM_MAX ; trans->used_mempool = true; -kfree (trans->mem); } -if (!new_mem) -return ERR_PTR (-BCH_ERR_ENOMEM_trans_kmalloc); +EBUG_ON (!new_mem); trans->mem = new_mem; trans->mem_bytes = new_bytes;
@@ -3247,14 +3254,6 @@ void *__bch2_trans_kmalloc(struct btree_trans *trans, size_t size, unsigned long trans->mem = new_mem; trans->mem_bytes = new_bytes; -if (old_bytes) { -trace_and_count (c, trans_restart_mem_realloced, trans, _RET_IP_, new_bytes); -return ERR_PTR (btree_trans_restart_ip (trans, - BCH_ERR_transaction_restart_mem_realloced, _RET_IP_)); - } - -bch2_trans_kmalloc_trace (trans, size, ip); - p = trans->mem + trans->mem_top; trans->mem_top += size;memset (p,0 , size);
@@ -3315,6 +3314,27 @@ u32 bch2_trans_begin(struct btree_trans *trans) trans->restart_count++; trans->mem_top =0 ; +if (trans->restarted == BCH_ERR_transaction_restart_mem_realloced) { +EBUG_ON (!trans->mem || !trans->mem_bytes); +unsigned new_bytes = trans->realloc_bytes_required; +void *new_mem =krealloc (trans->mem, new_bytes,GFP_NOWAIT |__GFP_NOWARN); +if (unlikely (!new_mem)) { +bch2_trans_unlock (trans); + new_mem =krealloc (trans->mem, new_bytes,GFP_KERNEL ); + +EBUG_ON (new_bytes >BTREE_TRANS_MEM_MAX ); + +if (!new_mem) { + new_mem =mempool_alloc (&trans->c->btree_trans_mem_pool,GFP_KERNEL ); + new_bytes =BTREE_TRANS_MEM_MAX ; + trans->used_mempool = true; +kfree (trans->mem); + } + } + trans->mem = new_mem; + trans->mem_bytes = new_bytes; + } +trans_for_each_path (trans, path, i) { path->should_be_locked = false;
fs/bcachefs/btree_types.h +1 -0
@@ -497,6 +497,7 @@ struct btree_trans {void *mem;unsigned mem_top;unsigned mem_bytes; +unsigned realloc_bytes_required;#ifdef CONFIG_BCACHEFS_TRANS_KMALLOC_TRACE darray_trans_kmalloc_trace trans_kmalloc_trace;#endif
Signature · 1 line
2.50.0
PATCH 5/37 bcachefs: Add missing EBUG_ON
In reply to Kent OverstreetFrom: Alan Huang <[email protected]>
Just like the EBUG_ON in bch2_journal_add_entry().
--- fs/bcachefs/btree_trans_commit.c | 2 ++ 1 file changed, 2 insertions(+)
fs/bcachefs/btree_trans_commit.c +2 -0
@@ -757,6 +757,8 @@ bch2_trans_commit_write_locked(struct btree_trans *trans, unsigned flags,btree_trans_journal_entries_start (trans), trans->journal_entries.u64s); +EBUG_ON (trans->journal_res.u64s < trans->journal_entries.u64s); + trans->journal_res.offset += trans->journal_entries.u64s; trans->journal_res.u64s -= trans->journal_entries.u64s;
Signature · 1 line
2.50.0
PATCH 6/37 bcachefs: Delay calculation of trans->journal_u64s
In reply to Kent OverstreetFrom: Alan Huang <[email protected]>
When there is commit error that need split btree leaf, fsck might change the value of trans->journal_entries.u64s, when retry commit, the value of trans->journal_u64s would be incorrect, which will lead to trans->journal_res.u64s underflow, and then out of bounds write will occur:
[ 464.496970][T11969] Call trace: [ 464.496973][T11969] show_stack+0x3c/0x88 (C) [ 464.496995][T11969] dump_stack_lvl+0xf8/0x178 [ 464.497014][T11969] dump_stack+0x20/0x30 [ 464.497031][T11969] __bch2_trans_log_str+0x344/0x350 [ 464.497048][T11969] bch2_trans_log_str+0x3c/0x60 [ 464.497065][T11969] __bch2_fsck_err+0x11bc/0x1390 [ 464.497083][T11969] bch2_check_discard_freespace_key+0xad4/0x10d0 [ 464.497100][T11969] bch2_bucket_alloc_freelist+0x99c/0x1130 [ 464.497117][T11969] bch2_bucket_alloc_trans+0x79c/0xcb8 [ 464.497133][T11969] bch2_bucket_alloc_set_trans+0x378/0xc20 [ 464.497151][T11969] __open_bucket_add_buckets+0x7fc/0x1c00 [ 464.497168][T11969] open_bucket_add_buckets+0x184/0x3a8 [ 464.497185][T11969] bch2_alloc_sectors_start_trans+0xa04/0x1da0 [ 464.497203][T11969] bch2_btree_reserve_get+0x6e0/0xef0 [ 464.497220][T11969] bch2_btree_update_start+0x1618/0x2600 [ 464.497239][T11969] bch2_btree_split_leaf+0xcc/0x730 [ 464.497258][T11969] bch2_trans_commit_error+0x22c/0xc30 [ 464.497276][T11969] __bch2_trans_commit+0x207c/0x4e30 [ 464.497292][T11969] bch2_journal_replay+0x9e0/0x1420 [ 464.497305][T11969] __bch2_run_recovery_passes+0x458/0xf98 [ 464.497318][T11969] bch2_run_recovery_passes+0x280/0x478 [ 464.497331][T11969] bch2_fs_recovery+0x24f0/0x3a28 [ 464.497344][T11969] bch2_fs_start+0xb80/0x1248 [ 464.497358][T11969] bch2_fs_get_tree+0xe94/0x1708 [ 464.497377][T11969] vfs_get_tree+0x84/0x2d0
--- fs/bcachefs/btree_trans_commit.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
fs/bcachefs/btree_trans_commit.c +7 -4
@@ -1005,6 +1005,7 @@ int __bch2_trans_commit(struct btree_trans *trans, unsigned flags) {struct btree_insert_entry *errored_at =NULL ;struct bch_fs *c = trans->c; +unsigned journal_u64s =0 ;int ret =0 ;bch2_trans_verify_not_unlocked_or_in_restart (trans);
@@ -1033,10 +1034,10 @@ int __bch2_trans_commit(struct btree_trans *trans, unsigned flags)EBUG_ON (test_bit (BCH_FS_clean_shutdown, &c->flags)); - trans->journal_u64s = trans->journal_entries.u64s +jset_u64s (trans->accounting.u64s); + journal_u64s =jset_u64s (trans->accounting.u64s); trans->journal_transaction_names =READ_ONCE (c->opts.journal_transaction_names);if (trans->journal_transaction_names) - trans->journal_u64s +=jset_u64s (JSET_ENTRY_LOG_U64s); + journal_u64s +=jset_u64s (JSET_ENTRY_LOG_U64s);trans_for_each_update (trans, i) {struct btree_path *path = trans->paths + i->path;
@@ -1056,11 +1057,11 @@ int __bch2_trans_commit(struct btree_trans *trans, unsigned flags)continue ;/* we're going to journal the key being updated: */ - trans->journal_u64s +=jset_u64s (i->k->k.u64s); + journal_u64s +=jset_u64s (i->k->k.u64s);/* and we're also going to log the overwrite: */ if (trans->journal_transaction_names) - trans->journal_u64s +=jset_u64s (i->old_k.u64s); + journal_u64s +=jset_u64s (i->old_k.u64s); }if (trans->extra_disk_res) {
@@ -1078,6 +1079,8 @@ int __bch2_trans_commit(struct btree_trans *trans, unsigned flags)memset (&trans->journal_res,0 ,sizeof (trans->journal_res));memset (&trans->fs_usage_delta,0 ,sizeof (trans->fs_usage_delta)); + trans->journal_u64s = journal_u64s + trans->journal_entries.u64s; + ret =do_bch2_trans_commit (trans, flags, &errored_at, _RET_IP_);/* make sure we didn't drop or screw up locks: */
Signature · 1 line
2.50.0
PATCH 7/37 bcachefs: Fix bch2_journal_keys_peek_prev_min()
In reply to Kent Overstreetthis code is rarely invoked, so - we had a few bugs left from basing it off of bch2_journal_keys_peek_max()...
--- fs/bcachefs/btree_journal_iter.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/bcachefs/btree_journal_iter.c +2 -2
@@ -141,8 +141,8 @@ struct bkey_i *bch2_journal_keys_peek_prev_min(struct bch_fs *c, enum btree_id bif (!*idx) *idx =__bch2_journal_key_search (keys, btree_id, level, pos); -while (*idx && -__journal_key_cmp (btree_id, level, end_pos,idx_to_key (keys, *idx -1 )) <=0 ) { +while (*idx < keys->nr && +__journal_key_cmp (btree_id, level, end_pos,idx_to_key (keys, *idx -1 )) >=0 ) { (*idx)++; iters++;if (iters ==10 ) {
Signature · 1 line
2.50.0
PATCH 8/37 bcachefs: btree_iter: fix updates, journal overlay
In reply to Kent OverstreetWe need to start searching from search_key - _not_ path->pos, which will point to the key we found in the btree
--- fs/bcachefs/btree_iter.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-)
fs/bcachefs/btree_iter.c +17 -12
@@ -2076,14 +2076,14 @@ inline bool bch2_btree_iter_rewind(struct btree_trans *trans, struct btree_iterstatic noinline void bch2_btree_trans_peek_prev_updates(struct btree_trans *trans, struct btree_iter *iter, -struct bkey_s_c *k) +struct bpos search_key,struct bkey_s_c *k) {struct bpos end =path_l (btree_iter_path (trans, iter))->b->data->min_key;trans_for_each_update (trans, i)if (!i->key_cache_already_flushed && i->btree_id == iter->btree_id && -bpos_le (i->k->k.p, iter->pos) && +bpos_le (i->k->k.p, search_key) &&bpos_ge (i->k->k.p, k->k ? k->k->p : end)) { iter->k = i->k->k; *k =bkey_i_to_s_c (i->k);
@@ -2092,6 +2092,7 @@ void bch2_btree_trans_peek_prev_updates(struct btree_trans *trans, struct btree_static noinline void bch2_btree_trans_peek_updates(struct btree_trans *trans, struct btree_iter *iter, +struct bpos search_key,struct bkey_s_c *k) {struct btree_path *path =btree_iter_path (trans, iter);
@@ -2100,7 +2101,7 @@ void bch2_btree_trans_peek_updates(struct btree_trans *trans, struct btree_itertrans_for_each_update (trans, i)if (!i->key_cache_already_flushed && i->btree_id == iter->btree_id && -bpos_ge (i->k->k.p, path->pos) && +bpos_ge (i->k->k.p, search_key) &&bpos_le (i->k->k.p, k->k ? k->k->p : end)) { iter->k = i->k->k; *k =bkey_i_to_s_c (i->k);
@@ -2122,13 +2123,14 @@ void bch2_btree_trans_peek_slot_updates(struct btree_trans *trans, struct btree_static struct bkey_i *bch2_btree_journal_peek (struct btree_trans *trans,struct btree_iter *iter, +struct bpos search_pos,struct bpos end_pos) {struct btree_path *path =btree_iter_path (trans, iter);return bch2_journal_keys_peek_max (trans->c, iter->btree_id, path->level, - path->pos, + search_pos, end_pos, &iter->journal_idx); }
@@ -2138,7 +2140,7 @@ struct bkey_s_c btree_trans_peek_slot_journal(struct btree_trans *trans, struct btree_iter *iter) {struct btree_path *path =btree_iter_path (trans, iter); -struct bkey_i *k =bch2_btree_journal_peek (trans, iter, path->pos); +struct bkey_i *k =bch2_btree_journal_peek (trans, iter, path->pos, path->pos);if (k) { iter->k = k->k;
@@ -2151,11 +2153,12 @@ struct bkey_s_c btree_trans_peek_slot_journal(struct btree_trans *trans,static noinline void btree_trans_peek_journal(struct btree_trans *trans,struct btree_iter *iter, +struct bpos search_key,struct bkey_s_c *k) {struct btree_path *path =btree_iter_path (trans, iter);struct bkey_i *next_journal = -bch2_btree_journal_peek (trans, iter, +bch2_btree_journal_peek (trans, iter, search_key, k->k ? k->k->p :path_l (path)->b->key.k.p);if (next_journal) { iter->k = next_journal->k;
@@ -2165,13 +2168,14 @@ void btree_trans_peek_journal(struct btree_trans *trans,static struct bkey_i *bch2_btree_journal_peek_prev (struct btree_trans *trans,struct btree_iter *iter, +struct bpos search_key,struct bpos end_pos) {struct btree_path *path =btree_iter_path (trans, iter);return bch2_journal_keys_peek_prev_min (trans->c, iter->btree_id, path->level, - path->pos, + search_key, end_pos, &iter->journal_idx); }
@@ -2179,11 +2183,12 @@ static struct bkey_i *bch2_btree_journal_peek_prev(struct btree_trans *trans,static noinline void btree_trans_peek_prev_journal(struct btree_trans *trans,struct btree_iter *iter, +struct bpos search_key,struct bkey_s_c *k) {struct btree_path *path =btree_iter_path (trans, iter);struct bkey_i *next_journal = -bch2_btree_journal_peek_prev (trans, iter, +bch2_btree_journal_peek_prev (trans, iter, search_key, k->k ? k->k->p :path_l (path)->b->key.k.p);if (next_journal) {
@@ -2292,11 +2297,11 @@ static struct bkey_s_c __bch2_btree_iter_peek(struct btree_trans *trans, struct }if (unlikely (iter->flags & BTREE_ITER_with_journal)) -btree_trans_peek_journal (trans, iter, &k); +btree_trans_peek_journal (trans, iter, search_key, &k);if (unlikely ((iter->flags & BTREE_ITER_with_updates) && trans->nr_updates)) -bch2_btree_trans_peek_updates (trans, iter, &k); +bch2_btree_trans_peek_updates (trans, iter, search_key, &k);if (k.k &&bkey_deleted (k.k)) {/*
@@ -2584,11 +2589,11 @@ static struct bkey_s_c __bch2_btree_iter_peek_prev(struct btree_trans *trans, st }if (unlikely (iter->flags & BTREE_ITER_with_journal)) -btree_trans_peek_prev_journal (trans, iter, &k); +btree_trans_peek_prev_journal (trans, iter, search_key, &k);if (unlikely ((iter->flags & BTREE_ITER_with_updates) && trans->nr_updates)) -bch2_btree_trans_peek_prev_updates (trans, iter, &k); +bch2_btree_trans_peek_prev_updates (trans, iter, search_key, &k);if (likely (k.k && !bkey_deleted (k.k))) {break ;
Signature · 1 line
2.50.0
PATCH 9/37 bcachefs: better __bch2_snapshot_is_ancestor() assert
In reply to Kent OverstreetPreviously, we weren't checking the result of the skiplist walk, just the is_ancestor bitmap.
--- fs/bcachefs/snapshot.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
fs/bcachefs/snapshot.c +5 -3
@@ -135,7 +135,9 @@ static bool test_ancestor_bitmap(struct snapshot_table *t, u32 id, u32 ancestor)bool __bch2_snapshot_is_ancestor (struct bch_fs *c,u32 id,u32 ancestor) { -bool ret; +#ifdef CONFIG_BCACHEFS_DEBUG +u32 orig_id = id; +#endif guard (rcu)();struct snapshot_table *t =rcu_dereference (c->snapshots);
@@ -147,11 +149,11 @@ bool __bch2_snapshot_is_ancestor(struct bch_fs *c, u32 id, u32 ancestor)while (id && id < ancestor -IS_ANCESTOR_BITMAP ) id =get_ancestor_below (t, id, ancestor); - ret = id && id < ancestor +bool ret = id && id < ancestor ?test_ancestor_bitmap (t, id, ancestor) : id == ancestor; -EBUG_ON (ret !=__bch2_snapshot_is_ancestor_early (t, id, ancestor)); +EBUG_ON (ret !=__bch2_snapshot_is_ancestor_early (t, orig_id, ancestor));return ret; }
Signature · 1 line
2.50.0
PATCH 10/37 bcachefs: pass last_seq into fs_journal_start()
In reply to Kent OverstreetPrep work for journal rewind, where the seq we're replaying from may be different than the last journal entry's last_seq.
--- fs/bcachefs/journal.c | 18 ++++++------------ fs/bcachefs/journal.h | 2 +- fs/bcachefs/recovery.c | 4 ++-- 3 files changed, 9 insertions(+), 15 deletions(-)
fs/bcachefs/journal.c +6 -12
@@ -1474,14 +1474,13 @@ void bch2_fs_journal_stop(struct journal *j)clear_bit (JOURNAL_running, &j->flags); } -int bch2_fs_journal_start (struct journal *j,u64 cur_seq) +int bch2_fs_journal_start (struct journal *j,u64 last_seq,u64 cur_seq) { struct bch_fs *c = container_of(j, struct bch_fs, journal);struct journal_entry_pin_list *p;struct journal_replay *i, **_i;struct genradix_iter iter;bool had_entries = false; -u64 last_seq = cur_seq, nr, seq;/* *
@@ -1495,17 +1494,11 @@ int bch2_fs_journal_start(struct journal *j, u64 cur_seq)return -EINVAL ; } -genradix_for_each_reverse (&c->journal_entries, iter, _i) { - i = *_i; - -if (journal_replay_ignore (i)) -continue ; +/* Clean filesystem? */ +if (!last_seq) + last_seq = cur_seq; - last_seq =le64_to_cpu (i->j.last_seq); -break ; - } - - nr = cur_seq - last_seq; +u64 nr = cur_seq - last_seq;/* * Extra fudge factor, in case we crashed when the journal pin fifo was
@@ -1532,6 +1525,7 @@ int bch2_fs_journal_start(struct journal *j, u64 cur_seq) j->pin.back = cur_seq;atomic64_set (&j->seq, cur_seq -1 ); +u64 seq;fifo_for_each_entry_ptr (p, &j->pin, seq)journal_pin_list_init (p,1 );
fs/bcachefs/journal.h +1 -1
@@ -453,7 +453,7 @@ int bch2_fs_journal_alloc(struct bch_fs *);void bch2_dev_journal_stop (struct journal *,struct bch_dev *);void bch2_fs_journal_stop (struct journal *); -int bch2_fs_journal_start (struct journal *,u64 ); +int bch2_fs_journal_start (struct journal *,u64 ,u64 );void bch2_journal_set_replay_done (struct journal *);void bch2_dev_journal_exit (struct bch_dev *);
fs/bcachefs/recovery.c +2 -2
@@ -965,7 +965,7 @@ int bch2_fs_recovery(struct bch_fs *c) ret =bch2_journal_log_msg (c,"starting journal at entry %llu, replaying %llu-%llu" , journal_seq, last_seq, blacklist_seq -1 ) ?: -bch2_fs_journal_start (&c->journal, journal_seq); +bch2_fs_journal_start (&c->journal, last_seq, journal_seq);if (ret) goto err;
@@ -1181,7 +1181,7 @@ int bch2_fs_initialize(struct bch_fs *c)* journal_res_get() will crash if called before this has * set up the journal.pin FIFO and journal.cur pointer: */ - ret =bch2_fs_journal_start (&c->journal,1 ); + ret =bch2_fs_journal_start (&c->journal,1 ,1 );if (ret) goto err;
Signature · 1 line
2.50.0
PATCH 11/37 bcachefs: Fix "now allowing incompatible features" message
In reply to Kent OverstreetCheck against version_incompat_allowed, not version_incompat.
--- fs/bcachefs/recovery.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/bcachefs/recovery.c +1 -1
@@ -692,7 +692,7 @@ static bool check_version_upgrade(struct bch_fs *c) ret = true; } -if (new_version > c->sb.version_incompat && +if (new_version > c->sb.version_incompat_allowed && c->opts.version_upgrade == BCH_VERSION_UPGRADE_incompatible) {struct printbuf buf =PRINTBUF ;
Signature · 1 line
2.50.0
PATCH 12/37 bcachefs: Fix snapshot_key_missing_inode_snapshot repair
In reply to Kent OverstreetWhen the inode was a whiteout, we were inserting a new whiteout at the wrong (old) snapshot.
--- fs/bcachefs/fsck.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
fs/bcachefs/fsck.c +4 -6
@@ -903,17 +903,15 @@ lookup_inode_for_snapshot(struct btree_trans *trans, struct inode_walker *w, str w->last_pos.inode, k.k->p.snapshot, i->inode.bi_snapshot, (bch2_bkey_val_to_text(&buf, c, k), buf.buf))) { -struct bch_inode_unpacked new = i->inode; -struct bkey_i whiteout; - - new.bi_snapshot = k.k->p.snapshot; -if (!i->whiteout) { +struct bch_inode_unpacked new = i->inode; + new.bi_snapshot = k.k->p.snapshot; ret =__bch2_fsck_write_inode (trans, &new); }else { +struct bkey_i whiteout;bkey_init (&whiteout.k); whiteout.k.type = KEY_TYPE_whiteout; - whiteout.k.p =SPOS (0 , i->inode.bi_inum, i->inode.bi_snapshot); + whiteout.k.p =SPOS (0 , i->inode.bi_inum, k.k->p.snapshot); ret =bch2_btree_insert_nonextent (trans, BTREE_ID_inodes, &whiteout, BTREE_UPDATE_internal_snapshot_node);
Signature · 1 line
2.50.0
PATCH 13/37 bcachefs: fsck: fix add_inode()
In reply to Kent Overstreetthe inode btree uses the offset field for the inum, not the inode field.
--- fs/bcachefs/fsck.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/bcachefs/fsck.c +1 -1
@@ -806,7 +806,7 @@ static int add_inode(struct bch_fs *c, struct inode_walker *w,if (!n->whiteout) {return bch2_inode_unpack (inode, &n->inode); }else { - n->inode.bi_inum = inode.k->p.inode; + n->inode.bi_inum = inode.k->p.offset; n->inode.bi_snapshot = inode.k->p.snapshot;return 0 ; }
Signature · 1 line
2.50.0
PATCH 14/37 bcachefs: fsck: fix extent past end of inode repair
In reply to Kent OverstreetFix the case where we're deleting in a different snapshot and need to emit a whiteout - that requires a regular BTREE_ITER_filter_snapshots iterator.
Also, only delete the part of the extent that extents past i_size.
--- fs/bcachefs/fsck.c | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-)
fs/bcachefs/fsck.c +26 -5
@@ -1820,18 +1820,39 @@ static int check_extent(struct btree_trans *trans, struct btree_iter *iter, !key_visible_in_snapshot(c, s, i->inode.bi_snapshot, k.k->p.snapshot)) continue; -if (fsck_err_on (k.k->p.offset >round_up (i->inode.bi_size,block_bytes (c)) >>9 && +u64 last_block =round_up (i->inode.bi_size,block_bytes (c)) >>9 ; + +if (fsck_err_on (k.k->p.offset > last_block && !bkey_extent_is_reservation (k), trans, extent_past_end_of_inode,"extent type past end of inode %llu:%u, i_size %llu\n%s" , i->inode.bi_inum, i->inode.bi_snapshot, i->inode.bi_size, (bch2_bkey_val_to_text (&buf, c, k), buf.buf))) { -struct btree_iter iter2; +struct bkey_i *whiteout =bch2_trans_kmalloc (trans,sizeof (*whiteout)); + ret =PTR_ERR_OR_ZERO (whiteout); +if (ret) + goto err; -bch2_trans_copy_iter (trans, &iter2, iter); -bch2_btree_iter_set_snapshot (trans, &iter2, i->inode.bi_snapshot); +bkey_init (&whiteout->k); + whiteout->k.p =SPOS (k.k->p.inode, + last_block, + i->inode.bi_snapshot); +bch2_key_resize (&whiteout->k, +min (KEY_SIZE_MAX & (~0 << c->block_bits), +U64_MAX - whiteout->k.p.offset)); + + +/* +* Need a normal (not BTREE_ITER_all_snapshots) +* iterator, if we're deleting in a different +* snapshot and need to emit a whiteout +*/ +struct btree_iter iter2; +bch2_trans_iter_init (trans, &iter2, BTREE_ID_extents, +bkey_start_pos (&whiteout->k), + BTREE_ITER_intent); ret =bch2_btree_iter_traverse (trans, &iter2) ?: -bch2_btree_delete_at (trans, &iter2, +bch2_trans_update (trans, &iter2, whiteout, BTREE_UPDATE_internal_snapshot_node);bch2_trans_iter_exit (trans, &iter2);if (ret)
Signature · 1 line
2.50.0
PATCH 15/37 bcachefs: opts.journal_rewind
In reply to Kent OverstreetAdd a mount option for rewinding the journal, bringing the entire filesystem to where it was at a previous point in time.
This is for extreme disaster recovery scenarios - it's not intended as an undelete operation.
The option takes a journal sequence number; the desired sequence number can be determined with 'bcachefs list_journal'
Caveats:
- The 'journal_transaction_names' option must have been enabled (it's on by default). The option controls emitting of extra debug info in the journal, so we can see what individual transactions were doing; It also enables journalling of keys being overwritten, which is what we rely on here.
- A full fsck run will be automatically triggered since alloc info will be inconsistent. Only leaf node updates to non-alloc btrees are rewound, since rewinding interior btree updates isn't possible or desirable.
- We can't do anything about data that was deleted and overwritten.
Lots of metadata updates after the point in time we're rewinding to shouldn't cause a problem, since we segragate data and metadata allocations (this is in order to make repair by btree node scan practical on larger filesystems; there's a small 64-bit per device bitmap in the superblock of device ranges with btree nodes, and we try to keep this small).
However, having discards enabled will cause problems, since buckets are discarded as soon as they become empty (this is why we don't implement fstrim: we don't need it).
Hopefully, this feature will be a one-off thing that's never used
again: this was implemented for recovering from the "vfs i_nlink 0 -> subvol deletion" bug, and that bug was unusually disastrous and additional safeguards have since been implemented.
But if it does turn out that we need this more in the future, I'll have to implement an option so that empty buckets aren't discarded immediately - lagging by perhaps 1% of device capacity.
--- fs/bcachefs/btree_journal_iter.c | 60 +++++++++++++++++--------- fs/bcachefs/btree_journal_iter_types.h | 5 ++- fs/bcachefs/journal_io.c | 21 +++++++-- fs/bcachefs/opts.h | 5 +++ fs/bcachefs/recovery.c | 5 +++ 5 files changed, 69 insertions(+), 27 deletions(-)
fs/bcachefs/btree_journal_iter.c +39 -21
@@ -641,10 +641,11 @@ static int journal_sort_key_cmp(const void *_l, const void *_r) {const struct journal_key *l = _l;const struct journal_key *r = _r; +int rewind = l->rewind && r->rewind ?-1 :1 ;return journal_key_cmp (l, r) ?: -cmp_int (l->journal_seq, r->journal_seq) ?: -cmp_int (l->journal_offset, r->journal_offset); + ((cmp_int (l->journal_seq, r->journal_seq) ?: +cmp_int (l->journal_offset, r->journal_offset)) * rewind); }void bch2_journal_keys_put (struct bch_fs *c)
@@ -713,6 +714,8 @@ int bch2_journal_keys_sort(struct bch_fs *c)struct journal_keys *keys = &c->journal_keys;size_t nr_read =0 ; +u64 rewind_seq = c->opts.journal_rewind ?:U64_MAX ; +genradix_for_each (&c->journal_entries, iter, _i) { i = *_i;
@@ -721,28 +724,43 @@ int bch2_journal_keys_sort(struct bch_fs *c)cond_resched (); -for_each_jset_key (k, entry, &i->j) { -struct journal_key n = (struct journal_key ) { - .btree_id = entry->btree_id, - .level = entry->level, - .k = k, - .journal_seq =le64_to_cpu (i->j.seq), - .journal_offset = k->_data - i->j._data, - }; - -if (darray_push (keys, n)) { -__journal_keys_sort (keys); - -if (keys->nr *8 > keys->size *7 ) { -bch_err (c,"Too many journal keys for slowpath; have %zu compacted, buf size %zu, processed %zu keys at seq %llu" , - keys->nr, keys->size, nr_read,le64_to_cpu (i->j.seq)); -return bch_err_throw (c, ENOMEM_journal_keys_sort); +vstruct_for_each (&i->j, entry) { +bool rewind = !entry->level && + !btree_id_is_alloc (entry->btree_id) && +le64_to_cpu (i->j.seq) >= rewind_seq; + +if (entry->type != (rewind + ? BCH_JSET_ENTRY_overwrite + : BCH_JSET_ENTRY_btree_keys)) +continue ; + +if (!rewind &&le64_to_cpu (i->j.seq) < c->journal_replay_seq_start) +continue ; + +jset_entry_for_each_key (entry, k) { +struct journal_key n = (struct journal_key ) { + .btree_id = entry->btree_id, + .level = entry->level, + .rewind = rewind, + .k = k, + .journal_seq =le64_to_cpu (i->j.seq), + .journal_offset = k->_data - i->j._data, + }; + +if (darray_push (keys, n)) { +__journal_keys_sort (keys); + +if (keys->nr *8 > keys->size *7 ) { +bch_err (c,"Too many journal keys for slowpath; have %zu compacted, buf size %zu, processed %zu keys at seq %llu" , + keys->nr, keys->size, nr_read,le64_to_cpu (i->j.seq)); +return bch_err_throw (c, ENOMEM_journal_keys_sort); + } + +BUG_ON (darray_push (keys, n)); } -BUG_ON (darray_push (keys, n)); + nr_read++; } - - nr_read++; } }
fs/bcachefs/btree_journal_iter_types.h +3 -2
@@ -11,8 +11,9 @@ struct journal_key {u32 journal_offset; enum btree_id btree_id:8; unsigned level:8; -bool allocated; -bool overwritten; + bool allocated:1; + bool overwritten:1; + bool rewind:1;struct journal_key_range_overwritten __rcu * overwritten_range;struct bkey_i *k;
fs/bcachefs/journal_io.c +17 -4
@@ -160,6 +160,9 @@ static int journal_entry_add(struct bch_fs *c, struct bch_dev *ca,struct printbuf buf =PRINTBUF ;int ret =JOURNAL_ENTRY_ADD_OK ; +if (last_seq && c->opts.journal_rewind) + last_seq =min (last_seq, c->opts.journal_rewind); +if (!c->journal.oldest_seq_found_ondisk ||le64_to_cpu (j->seq) < c->journal.oldest_seq_found_ondisk) c->journal.oldest_seq_found_ondisk =le64_to_cpu (j->seq);
@@ -1430,11 +1433,21 @@ int bch2_journal_read(struct bch_fs *c,printbuf_reset (&buf);prt_printf (&buf,"journal read done, replaying entries %llu-%llu" , *last_seq, *blacklist_seq -1 ); + +/* +* Drop blacklisted entries and entries older than last_seq (or start of +* journal rewind: +*/ +u64 drop_before = *last_seq; +if (c->opts.journal_rewind) { + drop_before =min (drop_before, c->opts.journal_rewind); +prt_printf (&buf," (rewinding from %llu)" , c->opts.journal_rewind); + } + + *last_seq = drop_before;if (*start_seq != *blacklist_seq)prt_printf (&buf," (unflushed %llu-%llu)" , *blacklist_seq, *start_seq -1 );bch_info (c,"%s" , buf.buf); - -/* Drop blacklisted entries and entries older than last_seq: */ genradix_for_each (&c->journal_entries, radix_iter, _i) { i = *_i;
@@ -1442,7 +1455,7 @@ int bch2_journal_read(struct bch_fs *c,continue ; seq =le64_to_cpu (i->j.seq); -if (seq < *last_seq) { +if (seq < drop_before) {journal_replay_free (c, i, false);continue ; }
@@ -1455,7 +1468,7 @@ int bch2_journal_read(struct bch_fs *c, } } - ret =bch2_journal_check_for_missing (c, *last_seq, *blacklist_seq -1 ); + ret =bch2_journal_check_for_missing (c, drop_before, *blacklist_seq -1 );if (ret) goto err;
fs/bcachefs/opts.h +5 -0
@@ -379,6 +379,11 @@ enum fsck_err_opts { OPT_BOOL(), \ BCH2_NO_SB_OPT, false, \ NULL, "Exit recovery immediately prior to journal replay")\ + x(journal_rewind, u64, \ + OPT_FS|OPT_MOUNT, \ + OPT_UINT(0, U64_MAX), \ + BCH2_NO_SB_OPT, 0, \ + NULL, "Rewind journal") \ x(recovery_passes, u64, \ OPT_FS|OPT_MOUNT, \ OPT_BITFIELD(bch2_recovery_passes), \
fs/bcachefs/recovery.c +5 -0
@@ -757,6 +757,11 @@ int bch2_fs_recovery(struct bch_fs *c)if (c->opts.nochanges) c->opts.read_only = true; +if (c->opts.journal_rewind) { +bch_info (c,"rewinding journal, fsck required" ); + c->opts.fsck = true; + } +mutex_lock (&c->sb_lock);struct bch_sb_field_ext *ext =bch2_sb_field_get (c->disk_sb.sb, ext);bool write_sb = false;
Signature · 1 line
2.50.0
PATCH 16/37 bcachefs: Kill unused tracepoints
In reply to Kent OverstreetDead code cleanup.
--- fs/bcachefs/btree_trans_commit.c | 5 +- fs/bcachefs/btree_write_buffer.c | 3 + fs/bcachefs/errcode.h | 5 -- fs/bcachefs/trace.h | 100 +------------------------------ 4 files changed, 9 insertions(+), 104 deletions(-)
fs/bcachefs/btree_trans_commit.c +3 -2
@@ -595,12 +595,13 @@ bch2_trans_commit_write_locked(struct btree_trans *trans, unsigned flags,int ret =0 ;bch2_trans_verify_not_unlocked_or_in_restart (trans); - +#if 0 +/* todo: bring back dynamic fault injection */ if (race_fault ()) {trace_and_count (c, trans_restart_fault_inject, trans, trace_ip);return btree_trans_restart (trans, BCH_ERR_transaction_restart_fault_inject); } - +#endif /* * Check if the insert will fit in the leaf node with the write lock * held, otherwise another thread could write the node changing the
fs/bcachefs/btree_write_buffer.c +3 -0
@@ -676,6 +676,9 @@ int bch2_btree_write_buffer_maybe_flush(struct btree_trans *trans, goto err;bch2_bkey_buf_copy (last_flushed, c, tmp.k); + +/* can we avoid the unconditional restart? */ +trace_and_count (c, trans_restart_write_buffer_flush, trans, _RET_IP_); ret =bch_err_throw (c, transaction_restart_write_buffer_flush); } err:
fs/bcachefs/errcode.h +0 -5
@@ -137,7 +137,6 @@ x(BCH_ERR_transaction_restart, transaction_restart_relock) \ x(BCH_ERR_transaction_restart, transaction_restart_relock_path) \ x(BCH_ERR_transaction_restart, transaction_restart_relock_path_intent) \ - x(BCH_ERR_transaction_restart, transaction_restart_relock_after_fill) \ x(BCH_ERR_transaction_restart, transaction_restart_too_many_iters) \ x(BCH_ERR_transaction_restart, transaction_restart_lock_node_reused) \x (BCH_ERR_transaction_restart, transaction_restart_fill_relock) \
@@ -148,11 +147,8 @@ x(BCH_ERR_transaction_restart, transaction_restart_would_deadlock_write)\ x(BCH_ERR_transaction_restart, transaction_restart_deadlock_recursion_limit)\ x(BCH_ERR_transaction_restart, transaction_restart_upgrade) \ - x(BCH_ERR_transaction_restart, transaction_restart_key_cache_upgrade) \ x(BCH_ERR_transaction_restart, transaction_restart_key_cache_fill) \ x(BCH_ERR_transaction_restart, transaction_restart_key_cache_raced) \ - x(BCH_ERR_transaction_restart, transaction_restart_key_cache_realloced)\ - x(BCH_ERR_transaction_restart, transaction_restart_journal_preres_get) \ x(BCH_ERR_transaction_restart, transaction_restart_split_race) \ x(BCH_ERR_transaction_restart, transaction_restart_write_buffer_flush) \x (BCH_ERR_transaction_restart, transaction_restart_nested) \
@@ -241,7 +237,6 @@ x(BCH_ERR_journal_res_blocked, journal_buf_enomem) \ x(BCH_ERR_journal_res_blocked, journal_stuck) \ x(BCH_ERR_journal_res_blocked, journal_retry_open) \ - x(BCH_ERR_journal_res_blocked, journal_preres_get_blocked) \ x(BCH_ERR_journal_res_blocked, bucket_alloc_blocked) \ x(BCH_ERR_journal_res_blocked, stripe_alloc_blocked) \x (BCH_ERR_invalid, invalid_sb) \
fs/bcachefs/trace.h +3 -97
141 lines not on this page · raw message
Signature · 1 line
2.50.0
PATCH 17/37 bcachefs: mark more errors autofix
In reply to Kent Overstreet--- fs/bcachefs/sb-errors_format.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
fs/bcachefs/sb-errors_format.h +4 -4
@@ -217,7 +217,7 @@ enum bch_fsck_flags { x(inode_str_hash_invalid, 194, 0) \ x(inode_v3_fields_start_bad, 195, 0) \ x(inode_snapshot_mismatch, 196, 0) \ - x(snapshot_key_missing_inode_snapshot, 314, 0) \ + x(snapshot_key_missing_inode_snapshot, 314, FSCK_AUTOFIX) \ x(inode_unlinked_but_clean, 197, 0) \ x(inode_unlinked_but_nlink_nonzero, 198, 0) \x (inode_unlinked_and_not_open,281 ,0 ) \
@@ -253,18 +253,18 @@ enum bch_fsck_flags { x(extent_overlapping, 215, 0) \ x(key_in_missing_inode, 216, 0) \ x(key_in_wrong_inode_type, 217, 0) \ - x(extent_past_end_of_inode, 218, 0) \ + x(extent_past_end_of_inode, 218, FSCK_AUTOFIX) \ x(dirent_empty_name, 219, 0) \ x(dirent_val_too_big, 220, 0) \ x(dirent_name_too_long, 221, 0) \ x(dirent_name_embedded_nul, 222, 0) \ x(dirent_name_dot_or_dotdot, 223, 0) \ x(dirent_name_has_slash, 224, 0) \ - x(dirent_d_type_wrong, 225, 0) \ + x(dirent_d_type_wrong, 225, FSCK_AUTOFIX) \ x(inode_bi_parent_wrong, 226, 0) \ x(dirent_in_missing_dir_inode, 227, 0) \ x(dirent_in_non_dir_inode, 228, 0) \ - x(dirent_to_missing_inode, 229, 0) \ + x(dirent_to_missing_inode, 229, FSCK_AUTOFIX) \ x(dirent_to_overwritten_inode, 302, 0) \ x(dirent_to_missing_subvol, 230, 0) \x (dirent_to_itself,231 ,0 ) \
Signature · 1 line
2.50.0
PATCH 18/37 bcachefs: Move bset size check before csum check
In reply to Kent OverstreetFrom: Alan Huang <[email protected]>
In syzbot's crash, the bset's u64s is larger than the btree node.
--- fs/bcachefs/btree_io.c | 42 ++++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 20 deletions(-)
fs/bcachefs/btree_io.c +22 -20
@@ -723,12 +723,11 @@ void bch2_btree_node_drop_keys_outside_node(struct btree *b)static int validate_bset (struct bch_fs *c,struct bch_dev *ca,struct btree *b,struct bset *i, -unsigned offset,unsigned sectors,int write, +unsigned offset,int write,struct bch_io_failures *failed,struct printbuf *err_msg) {unsigned version =le16_to_cpu (i->version); -unsigned ptr_written =btree_ptr_sectors_written (bkey_i_to_s_c (&b->key));struct printbuf buf1 =PRINTBUF ;struct printbuf buf2 =PRINTBUF ;int ret =0 ;
@@ -778,15 +777,6 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, btree_node_unsupported_version, "BSET_SEPARATE_WHITEOUTS no longer supported"); -if (!write && -btree_err_on (offset + sectors > (ptr_written ?:btree_sectors (c)), - -BCH_ERR_btree_node_read_err_fixable, - c, ca, b, i,NULL , - bset_past_end_of_btree_node, -"bset past end of btree node (offset %u len %u but written %zu)" , - offset, sectors, ptr_written ?:btree_sectors (c))) - i->u64s =0 ; - btree_err_on(offset && !i->u64s, -BCH_ERR_btree_node_read_err_fixable, c, ca, b, i, NULL,
@@ -1151,6 +1141,14 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, "unknown checksum type %llu", BSET_CSUM_TYPE(i));if (first) { + sectors =vstruct_sectors (b->data, c->block_bits); +if (btree_err_on (b->written + sectors > (ptr_written ?:btree_sectors (c)), + -BCH_ERR_btree_node_read_err_fixable, + c, ca, b, i,NULL , + bset_past_end_of_btree_node, +"bset past end of btree node (offset %u len %u but written %zu)" , + b->written, sectors, ptr_written ?:btree_sectors (c))) + i->u64s =0 ;if (good_csum_type) {struct bch_csum csum =csum_vstruct (c,BSET_CSUM_TYPE (i), nonce, b->data);bool csum_bad =bch2_crc_cmp (b->data->csum, csum);
@@ -1178,9 +1176,15 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, c,NULL , b,NULL ,NULL , btree_node_unsupported_version, "btree node does not have NEW_EXTENT_OVERWRITE set"); - - sectors =vstruct_sectors (b->data, c->block_bits); }else { + sectors =vstruct_sectors (bne, c->block_bits); +if (btree_err_on (b->written + sectors > (ptr_written ?:btree_sectors (c)), + -BCH_ERR_btree_node_read_err_fixable, + c, ca, b, i,NULL , + bset_past_end_of_btree_node, +"bset past end of btree node (offset %u len %u but written %zu)" , + b->written, sectors, ptr_written ?:btree_sectors (c))) + i->u64s =0 ;if (good_csum_type) {struct bch_csum csum =csum_vstruct (c,BSET_CSUM_TYPE (i), nonce, bne);bool csum_bad =bch2_crc_cmp (bne->csum, csum);
@@ -1201,14 +1205,12 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, "decrypting btree node: %s", bch2_err_str(ret))) goto fsck_err; } - - sectors =vstruct_sectors (bne, c->block_bits); } b->version_ondisk =min (b->version_ondisk,le16_to_cpu (i->version)); - ret =validate_bset (c, ca, b, i, b->written, sectors,READ , failed, err_msg); + ret =validate_bset (c, ca, b, i, b->written,READ , failed, err_msg);if (ret) goto fsck_err;
@@ -2267,7 +2269,7 @@ static void btree_node_write_endio(struct bio *bio) }static int validate_bset_for_write (struct bch_fs *c,struct btree *b, -struct bset *i,unsigned sectors) +struct bset *i) { int ret = bch2_bkey_validate(c, bkey_i_to_s_c(&b->key), (struct bkey_validate_context) {
@@ -2282,7 +2284,7 @@ static int validate_bset_for_write(struct bch_fs *c, struct btree *b, } ret =validate_bset_keys (c, b, i,WRITE ,NULL ,NULL ) ?: -validate_bset (c,NULL , b, i, b->written, sectors,WRITE ,NULL ,NULL ); +validate_bset (c,NULL , b, i, b->written,WRITE ,NULL ,NULL );if (ret) {bch2_inconsistent_error (c);dump_stack ();
@@ -2475,7 +2477,7 @@ void __bch2_btree_node_write(struct bch_fs *c, struct btree *b, unsigned flags)/* if we're going to be encrypting, check metadata validity first: */ if (validate_before_checksum && -validate_bset_for_write (c, b, i, sectors_to_write)) +validate_bset_for_write (c, b, i)) goto err; ret =bset_encrypt (c, i, b->written <<9 );
@@ -2492,7 +2494,7 @@ void __bch2_btree_node_write(struct bch_fs *c, struct btree *b, unsigned flags)/* if we're not encrypting, check metadata after checksumming: */ if (!validate_before_checksum && -validate_bset_for_write (c, b, i, sectors_to_write)) +validate_bset_for_write (c, b, i)) goto err;/*
Signature · 1 line
2.50.0
PATCH 19/37 bcachefs: Fix pool->alloc NULL pointer dereference
In reply to Kent OverstreetFrom: Alan Huang <[email protected]>
btree_interior_update_pool has not been initialized before the filesystem becomes read-write, thus mempool_alloc in bch2_btree_update_start will trigger pool->alloc NULL pointer dereference in mempool_alloc_noprof
--- fs/bcachefs/bcachefs.h | 3 ++- fs/bcachefs/chardev.c | 29 ++++++++++++++++++++++------- 2 files changed, 24 insertions(+), 8 deletions(-)
fs/bcachefs/bcachefs.h +2 -1
@@ -767,7 +767,8 @@ struct btree_trans_buf {x (sysfs) \x (btree_write_buffer ) \x (btree_node_scrub) \ -x (async_recovery_passes) +x (async_recovery_passes) \ +x (ioctl_data) enum bch_write_ref {#define x (n) BCH_WRITE_REF_##n,
fs/bcachefs/chardev.c +22 -7
@@ -319,6 +319,7 @@ static int bch2_data_thread(void *arg) ctx->stats.ret = BCH_IOCTL_DATA_EVENT_RET_done; ctx->stats.data_type = (int ) DATA_PROGRESS_DATA_TYPE_done; } +enumerated_ref_put (&ctx->c->writes, BCH_WRITE_REF_ioctl_data);return 0 ; }
@@ -378,15 +379,24 @@ static long bch2_ioctl_data(struct bch_fs *c,struct bch_data_ctx *ctx;int ret; -if (!capable (CAP_SYS_ADMIN )) -return -EPERM ; +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_ioctl_data)) +return -EROFS ; -if (arg.op >=BCH_DATA_OP_NR || arg.flags) -return -EINVAL ; +if (!capable (CAP_SYS_ADMIN )) { + ret = -EPERM ; + goto put_ref; + } + +if (arg.op >=BCH_DATA_OP_NR || arg.flags) { + ret = -EINVAL ; + goto put_ref; + } ctx =kzalloc (sizeof (*ctx),GFP_KERNEL ); -if (!ctx) -return -ENOMEM ; +if (!ctx) { + ret = -ENOMEM ; + goto put_ref; + } ctx->c = c; ctx->arg = arg;
@@ -395,7 +405,12 @@ static long bch2_ioctl_data(struct bch_fs *c, &bcachefs_data_ops, bch2_data_thread);if (ret <0 ) -kfree (ctx); + goto cleanup; +return ret; +cleanup: +kfree (ctx); +put_ref: +enumerated_ref_put (&c->writes, BCH_WRITE_REF_ioctl_data);return ret; }
Signature · 1 line
2.50.0
PATCH 20/37 bcachefs: don't return fsck_fix for unfixable node errors in __btree_err
In reply to Kent OverstreetFrom: Bharadwaj Raju <[email protected]>
After cd3cdb1ef706 ("Single err message for btree node reads"), all errors caused __btree_err to return -BCH_ERR_fsck_fix no matter what the actual error type was if the recovery pass was scanning for btree nodes. This lead to the code continuing despite things like bad node formats when they earlier would have caused a jump to fsck_err, because btree_err only jumps when the return from __btree_err does not match fsck_fix. Ultimately this lead to undefined behavior by attempting to unpack a key based on an invalid format.
Make only errors of type -BCH_ERR_btree_node_read_err_fixable cause __btree_err to return -BCH_ERR_fsck_fix when scanning for btree nodes.
--- fs/bcachefs/btree_io.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
fs/bcachefs/btree_io.c +3 -1
@@ -557,7 +557,9 @@ static int __btree_err(int ret, const char *fmt, ...) {if (c->recovery.curr_pass == BCH_RECOVERY_PASS_scan_for_btree_nodes) -return bch_err_throw (c, fsck_fix); +return ret == -BCH_ERR_btree_node_read_err_fixable + ?bch_err_throw (c, fsck_fix) + : ret;bool have_retry = false;int ret2;
Signature · 1 line
2.50.0
PATCH 21/37 bcachefs: fsck: Improve check_key_has_inode()
In reply to Kent OverstreetPrint out more info when we find a key (extent, dirent, xattr) for a missing inode - was there a good inode in an older snapshot, full(ish) list of keys for that missing inode, so we can make better decisions on how to repair.
If it looks like it should've been deleted, autofix it. If we ever hit the non-autofix cases, we'll want to write more repair code (possibly reconstituting the inode).
--- fs/bcachefs/fsck.c | 108 ++++++++++++++++++++++++++------- fs/bcachefs/sb-errors_format.h | 2 +- 2 files changed, 88 insertions(+), 22 deletions(-)
fs/bcachefs/fsck.c +87 -21
@@ -1436,6 +1436,7 @@ static int check_key_has_inode(struct btree_trans *trans, {struct bch_fs *c = trans->c;struct printbuf buf =PRINTBUF ; +struct btree_iter iter2 = {};int ret =PTR_ERR_OR_ZERO (i);if (ret)return ret;
@@ -1445,40 +1446,105 @@ static int check_key_has_inode(struct btree_trans *trans,bool have_inode = i && !i->whiteout; -if (!have_inode && (c->sb.btrees_lost_data &BIT_ULL (BTREE_ID_inodes))) { - ret =reconstruct_inode (trans, iter->btree_id, k.k->p.snapshot, k.k->p.inode) ?: -bch2_trans_commit (trans,NULL ,NULL , BCH_TRANS_COMMIT_no_enospc); -if (ret) - goto err; +if (!have_inode && (c->sb.btrees_lost_data &BIT_ULL (BTREE_ID_inodes))) + goto reconstruct; - inode->last_pos.inode--; - ret =bch_err_throw (c, transaction_restart_nested); - goto err; +if (have_inode &&btree_matches_i_mode (iter->btree_id, i->inode.bi_mode)) + goto out; + +prt_printf (&buf,", " ); + +bool have_old_inode = false; +darray_for_each (inode->inodes, i2) +if (!i2->whiteout && +bch2_snapshot_is_ancestor (c, k.k->p.snapshot, i2->inode.bi_snapshot) && +btree_matches_i_mode (iter->btree_id, i2->inode.bi_mode)) { +prt_printf (&buf,"but found good inode in older snapshot\n" ); +bch2_inode_unpacked_to_text (&buf, &i2->inode); +prt_newline (&buf); + have_old_inode = true; +break ; + } + +struct bkey_s_c k2; +unsigned nr_keys =0 ; + +prt_printf (&buf,"found keys:\n" ); + +for_each_btree_key_max_norestart (trans, iter2, iter->btree_id, +SPOS (k.k->p.inode,0 , k.k->p.snapshot), +POS (k.k->p.inode,U64_MAX ), +0 , k2, ret) { + nr_keys++; +if (nr_keys <=10 ) { +bch2_bkey_val_to_text (&buf, c, k2); +prt_newline (&buf); + } +if (nr_keys >=100 ) +break ; } -if (fsck_err_on (!have_inode, - trans, key_in_missing_inode, -"key in missing inode:\n%s" , - (printbuf_reset (&buf), -bch2_bkey_val_to_text (&buf, c, k), buf.buf))) - goto delete; +if (ret) + goto err; -if (fsck_err_on (have_inode && !btree_matches_i_mode (iter->btree_id, i->inode.bi_mode), - trans, key_in_wrong_inode_type, -"key for wrong inode mode %o:\n%s" , - i->inode.bi_mode, - (printbuf_reset (&buf), -bch2_bkey_val_to_text (&buf, c, k), buf.buf))) - goto delete; +if (nr_keys >100 ) +prt_printf (&buf,"found > %u keys for this missing inode\n" , nr_keys); +else if (nr_keys >10 ) +prt_printf (&buf,"found %u keys for this missing inode\n" , nr_keys); + +if (!have_inode) { +if (fsck_err_on (!have_inode, + trans, key_in_missing_inode, +"key in missing inode%s" , buf.buf)) { +/* +* Maybe a deletion that raced with data move, or something +* weird like that? But if we know the inode was deleted, or +* it's just a few keys, we can safely delete them. +* +* If it's many keys, we should probably recreate the inode +*/ +if (have_old_inode || nr_keys <=2 ) + goto delete; +else + goto reconstruct; + } + }else { +/* +* not autofix, this one would be a giant wtf - bit error in the +* inode corrupting i_mode? +* +* may want to try repairing inode instead of deleting +*/ +if (fsck_err_on (!btree_matches_i_mode (iter->btree_id, i->inode.bi_mode), + trans, key_in_wrong_inode_type, +"key for wrong inode mode %o%s" , + i->inode.bi_mode, buf.buf)) + goto delete; + } out: err: fsck_err: +bch2_trans_iter_exit (trans, &iter2);printbuf_exit (&buf);bch_err_fn (c, ret);return ret; delete: +/* +* XXX: print out more info +* count up extents for this inode, check if we have different inode in +* an older snapshot version, perhaps decide if we want to reconstitute +*/ ret =bch2_btree_delete_at (trans, iter, BTREE_UPDATE_internal_snapshot_node); goto out; +reconstruct: + ret =reconstruct_inode (trans, iter->btree_id, k.k->p.snapshot, k.k->p.inode) ?: +bch2_trans_commit (trans,NULL ,NULL , BCH_TRANS_COMMIT_no_enospc); +if (ret) + goto err; + + inode->last_pos.inode--; + ret =bch_err_throw (c, transaction_restart_nested); + goto out; }static int check_i_sectors_notnested (struct btree_trans *trans,struct inode_walker *w)
fs/bcachefs/sb-errors_format.h +1 -1
@@ -251,7 +251,7 @@ enum bch_fsck_flags { x(deleted_inode_not_unlinked, 214, FSCK_AUTOFIX) \ x(deleted_inode_has_child_snapshots, 288, FSCK_AUTOFIX) \ x(extent_overlapping, 215, 0) \ - x(key_in_missing_inode, 216, 0) \ + x(key_in_missing_inode, 216, FSCK_AUTOFIX) \ x(key_in_wrong_inode_type, 217, 0) \ x(extent_past_end_of_inode, 218, FSCK_AUTOFIX) \x (dirent_empty_name,219 ,0 ) \
Signature · 1 line
2.50.0
PATCH 22/37 bcachefs: Call bch2_fs_init_rw() early if we'll be going rw
In reply to Kent Overstreetkthread creation checks for pending signals, which is _very_ annoying if we have to do a long recovery and don't go rw until we've done significant work.
Check if we'll be going rw and pre-allocate kthreads/workqueues.
--- fs/bcachefs/recovery.c | 10 ++++++++++ fs/bcachefs/recovery_passes.c | 6 +----- fs/bcachefs/recovery_passes.h | 9 +++++++++ fs/bcachefs/super.c | 13 +++++++++++-- fs/bcachefs/super.h | 1 + 5 files changed, 32 insertions(+), 7 deletions(-)
fs/bcachefs/recovery.c +10 -0
@@ -762,6 +762,16 @@ int bch2_fs_recovery(struct bch_fs *c) c->opts.fsck = true; } +if (go_rw_in_recovery (c)) { +/* +* start workqueues/kworkers early - kthread creation checks for +* pending signals, which is _very_ annoying +*/ + ret =bch2_fs_init_rw (c); +if (ret) + goto err; + } +mutex_lock (&c->sb_lock);struct bch_sb_field_ext *ext =bch2_sb_field_get (c->disk_sb.sb, ext);bool write_sb = false;
fs/bcachefs/recovery_passes.c +1 -5
@@ -217,11 +217,7 @@ static int bch2_set_may_go_rw(struct bch_fs *c)set_bit (BCH_FS_may_go_rw, &c->flags); -if (keys->nr || - !c->opts.read_only || - !c->sb.clean || - c->opts.recovery_passes || - (c->opts.fsck && !(c->sb.features &BIT_ULL (BCH_FEATURE_no_alloc_info)))) { +if (go_rw_in_recovery (c)) {if (c->sb.features &BIT_ULL (BCH_FEATURE_no_alloc_info)) {bch_info (c,"mounting a filesystem with no alloc info read-write; will recreate" );bch2_reconstruct_alloc (c);
fs/bcachefs/recovery_passes.h +9 -0
@@ -17,6 +17,15 @@ enum bch_run_recovery_pass_flags { RUN_RECOVERY_PASS_ratelimit = BIT(1), }; +static inline bool go_rw_in_recovery (struct bch_fs *c) +{ +return (c->journal_keys.nr || + !c->opts.read_only || + !c->sb.clean || + c->opts.recovery_passes || + (c->opts.fsck && !(c->sb.features &BIT_ULL (BCH_FEATURE_no_alloc_info)))); +} +int bch2_run_print_explicit_recovery_pass (struct bch_fs *,enum bch_recovery_pass ); int __bch2_run_explicit_recovery_pass(struct bch_fs *, struct printbuf *,
fs/bcachefs/super.c +11 -2
@@ -210,7 +210,6 @@ static int bch2_dev_alloc(struct bch_fs *, unsigned);static int bch2_dev_sysfs_online (struct bch_fs *,struct bch_dev *);static void bch2_dev_io_ref_stop (struct bch_dev *,int );static void __bch2_dev_read_only (struct bch_fs *,struct bch_dev *); -static int bch2_fs_init_rw (struct bch_fs *);struct bch_fs *bch2_dev_to_fs (dev_t dev) {
@@ -794,7 +793,7 @@ static int bch2_fs_online(struct bch_fs *c)return ret; } -static int bch2_fs_init_rw (struct bch_fs *c) +int bch2_fs_init_rw (struct bch_fs *c) {if (test_bit (BCH_FS_rw_init_done, &c->flags))return 0 ;
@@ -1015,6 +1014,16 @@ static struct bch_fs *bch2_fs_alloc(struct bch_sb *sb, struct bch_opts *opts,if (ret) goto err; +if (go_rw_in_recovery (c)) { +/* +* start workqueues/kworkers early - kthread creation checks for +* pending signals, which is _very_ annoying +*/ + ret =bch2_fs_init_rw (c); +if (ret) + goto err; + } +#ifdef CONFIG_UNICODE /* Default encoding until we can potentially have more as an option. */ c->cf_encoding =utf8_load (BCH_FS_DEFAULT_UTF8_ENCODING );
fs/bcachefs/super.h +1 -0
@@ -46,6 +46,7 @@ void __bch2_fs_stop(struct bch_fs *);void bch2_fs_free (struct bch_fs *);void bch2_fs_stop (struct bch_fs *); +int bch2_fs_init_rw (struct bch_fs *);int bch2_fs_start (struct bch_fs *);struct bch_fs *bch2_fs_open (darray_const_str *,struct bch_opts *);
Signature · 1 line
2.50.0
PATCH 23/37 bcachefs: Fix __bch2_inum_to_path() when crossing subvol boundaries
In reply to Kent OverstreetThe bch2_subvolume_get_snapshot() call needs to happen before the dirent lookup - the dirent is in the parent subvolume.
Also, check for loops.
--- fs/bcachefs/namei.c | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-)
fs/bcachefs/namei.c +22 -5
@@ -625,14 +625,26 @@ static int __bch2_inum_to_path(struct btree_trans *trans, {unsigned orig_pos = path->pos;int ret =0 ; +DARRAY (subvol_inum) inums = {}; + +if (!snapshot) { + ret =bch2_subvolume_get_snapshot (trans, subvol, &snapshot); +if (ret) + goto disconnected; + }while (true) { -if (!snapshot) { - ret =bch2_subvolume_get_snapshot (trans, subvol, &snapshot); -if (ret) - goto disconnected; +subvol_inum n = (subvol_inum ) { subvol ?: snapshot, inum }; + +if (darray_find_p (inums, i, i->subvol == n.subvol && i->inum == n.inum)) { +prt_str_reversed (path,"(loop)" ); +break ; } + ret =darray_push (&inums, n); +if (ret) + goto err; +struct bch_inode_unpacked inode; ret =bch2_inode_find_by_inum_snapshot (trans, inum, snapshot, &inode,0 );if (ret)
@@ -650,7 +662,9 @@ static int __bch2_inum_to_path(struct btree_trans *trans, inum = inode.bi_dir;if (inode.bi_parent_subvol) { subvol = inode.bi_parent_subvol; - snapshot =0 ; + ret =bch2_subvolume_get_snapshot (trans, inode.bi_parent_subvol, &snapshot); +if (ret) + goto disconnected; }struct btree_iter d_iter;
@@ -662,6 +676,7 @@ static int __bch2_inum_to_path(struct btree_trans *trans, goto disconnected;struct qstr dirent_name =bch2_dirent_get_name (d); +prt_bytes_reversed (path, dirent_name.name, dirent_name.len);prt_char (path,'/' );
@@ -677,8 +692,10 @@ static int __bch2_inum_to_path(struct btree_trans *trans, goto err;reverse_bytes (path->buf + orig_pos, path->pos - orig_pos); +darray_exit (&inums);return 0 ; err: +darray_exit (&inums);return ret; disconnected:if (bch2_err_matches (ret, BCH_ERR_transaction_restart))
Signature · 1 line
2.50.0
PATCH 24/37 bcachefs: fsck: Print path when we find a subvol loop
In reply to Kent Overstreet--- fs/bcachefs/fsck.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
fs/bcachefs/fsck.c +9 -4
@@ -2578,6 +2578,11 @@ static int check_subvol_path(struct btree_trans *trans, struct btree_iter *iter,if (k.k->type != KEY_TYPE_subvolume)return 0 ; +subvol_inum start = { + .subvol = k.k->p.offset, + .inum =le64_to_cpu (bkey_s_c_to_subvolume (k).v->inode), + }; +while (k.k->p.offset !=BCACHEFS_ROOT_SUBVOL ) { ret =darray_push (&subvol_path, k.k->p.offset);if (ret)
@@ -2596,11 +2601,11 @@ static int check_subvol_path(struct btree_trans *trans, struct btree_iter *iter,if (darray_u32_has (&subvol_path, parent)) {printbuf_reset (&buf); -prt_printf (&buf,"subvolume loop:\n" ); +prt_printf (&buf,"subvolume loop: " ); -darray_for_each_reverse (subvol_path, i) -prt_printf (&buf,"%u " , *i); -prt_printf (&buf,"%u" , parent); + ret =bch2_inum_to_path (trans, start, &buf); +if (ret) + goto err;if (fsck_err (trans, subvol_loop,"%s" , buf.buf)) ret =reattach_subvol (trans, s);
Signature · 1 line
2.50.0
PATCH 25/37 bcachefs: fsck: Fix remove_backpointer() for subvol roots
In reply to Kent OverstreetThe dirent will be in a different snapshot if the inode is a subvolume root.
--- fs/bcachefs/fsck.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
fs/bcachefs/fsck.c +9 -1
@@ -493,10 +493,18 @@ static int remove_backpointer(struct btree_trans *trans,if (!inode->bi_dir)return 0 ; +u32 snapshot = inode->bi_snapshot; + +if (inode->bi_parent_subvol) { +int ret =bch2_subvolume_get_snapshot (trans, inode->bi_parent_subvol, &snapshot); +if (ret) +return ret; + } +struct bch_fs *c = trans->c;struct btree_iter iter;struct bkey_s_c_dirent d =dirent_get_by_pos (trans, &iter, -SPOS (inode->bi_dir, inode->bi_dir_offset, inode->bi_snapshot)); +SPOS (inode->bi_dir, inode->bi_dir_offset, snapshot));int ret =bkey_err (d) ?:dirent_points_to_inode (c, d, inode) ?:bch2_fsck_remove_dirent (trans, d.k->p);
Signature · 1 line
2.50.0
PATCH 26/37 bcachefs: fsck: Fix reattach_inode() for subvol roots
In reply to Kent Overstreetbch_subvolume.fs_path_parent needs to be updated as well, it should match inode.bi_parent_subvol.
--- fs/bcachefs/fsck.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
fs/bcachefs/fsck.c +12 -0
@@ -372,6 +372,18 @@ static int reattach_inode(struct btree_trans *trans, struct bch_inode_unpacked *if (inode->bi_subvol) { inode->bi_parent_subvol =BCACHEFS_ROOT_SUBVOL ; +struct btree_iter subvol_iter; +struct bkey_i_subvolume *subvol = +bch2_bkey_get_mut_typed (trans, &subvol_iter, + BTREE_ID_subvolumes,POS (0 , inode->bi_subvol), +0 , subvolume); + ret =PTR_ERR_OR_ZERO (subvol); +if (ret) +return ret; + + subvol->v.fs_path_parent =BCACHEFS_ROOT_SUBVOL ; +bch2_trans_iter_exit (trans, &subvol_iter); +u64 root_inum; ret =subvol_lookup (trans, inode->bi_parent_subvol, &dirent_snapshot, &root_inum);
Signature · 1 line
2.50.0
PATCH 27/37 bcachefs: fsck: check_directory_structure runs in reverse order
In reply to Kent OverstreetWhen we find a directory connectivity problem, we should do the repair in the oldest snapshot that has the issue - so that we don't end up duplicating work or making a real mess of things.
Oldest snapshot IDs have the highest integer value, so - just walk inodes in reverse order.
--- fs/bcachefs/fsck.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/bcachefs/fsck.c +1 -1
@@ -2841,7 +2841,7 @@ static int check_path_loop(struct btree_trans *trans, struct bkey_s_c inode_k)int bch2_check_directory_structure (struct bch_fs *c) { int ret = bch2_trans_run(c, - for_each_btree_key_commit(trans, iter, BTREE_ID_inodes, POS_MIN, + for_each_btree_key_reverse_commit(trans, iter, BTREE_ID_inodes, POS_MIN, BTREE_ITER_intent| BTREE_ITER_prefetch| BTREE_ITER_all_snapshots, k,
Signature · 1 line
2.50.0
PATCH 28/37 bcachefs: fsck: additional diagnostics for reattach_inode()
In reply to Kent OverstreetLog the inode's new path.
--- fs/bcachefs/fsck.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
fs/bcachefs/fsck.c +12 -0
@@ -399,6 +399,8 @@ static int reattach_inode(struct btree_trans *trans, struct bch_inode_unpacked *if (ret)return ret; +bch_verbose (c,"got lostfound inum %llu" , lostfound.bi_inum); + lostfound.bi_nlink +=S_ISDIR (inode->bi_mode);/* ensure lost+found inode is also present in inode snapshot */
@@ -435,6 +437,16 @@ static int reattach_inode(struct btree_trans *trans, struct bch_inode_unpacked *if (ret)return ret; + { +CLASS (printbuf, buf)(); + ret =bch2_inum_snapshot_to_path (trans, inode->bi_inum, + inode->bi_snapshot,NULL , &buf); +if (ret) +return ret; + +bch_info (c,"reattached at %s" , buf.buf); + } +/* * Fix up inodes in child snapshots: if they should also be reattached * update the backpointer field, if they should not be we need to emit
Signature · 1 line
2.50.0
PATCH 29/37 bcachefs: fsck: check_subdir_count logs path
In reply to Kent OverstreetWe can easily go from inode number -> path now, which makes for more useful log messages.
--- fs/bcachefs/fsck.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
fs/bcachefs/fsck.c +16 -8
@@ -2066,14 +2066,22 @@ static int check_subdir_count_notnested(struct btree_trans *trans, struct inode_continue ; } -if (fsck_err_on (i->inode.bi_nlink != i->count, - trans, inode_dir_wrong_nlink, -"directory %llu:%u with wrong i_nlink: got %u, should be %llu" , - w->last_pos.inode, i->inode.bi_snapshot, i->inode.bi_nlink, i->count)) { - i->inode.bi_nlink = i->count; - ret =bch2_fsck_write_inode (trans, &i->inode); -if (ret) -break ; +if (i->inode.bi_nlink != i->count) { +CLASS (printbuf, buf)(); + +lockrestart_do (trans, +bch2_inum_snapshot_to_path (trans, w->last_pos.inode, + i->inode.bi_snapshot,NULL , &buf)); + +if (fsck_err_on (i->inode.bi_nlink != i->count, + trans, inode_dir_wrong_nlink, +"directory with wrong i_nlink: got %u, should be %llu\n%s" , + i->inode.bi_nlink, i->count, buf.buf)) { + i->inode.bi_nlink = i->count; + ret =bch2_fsck_write_inode (trans, &i->inode); +if (ret) +break ; + } } } fsck_err:
Signature · 1 line
2.50.0
PATCH 30/37 bcachefs: fsck: Fix check_path_loop() + snapshots
In reply to Kent OverstreetA path exists in a particular snapshot: we should do the pathwalk in the snapshot ID of the inode we started from, _not_ change snapshot ID as we walk inodes and dirents.
--- fs/bcachefs/fsck.c | 62 +++++++++++++++++++--------------------------- 1 file changed, 26 insertions(+), 36 deletions(-)
fs/bcachefs/fsck.c +26 -36
@@ -2689,19 +2689,13 @@ int bch2_check_subvolume_structure(struct bch_fs *c)return ret; } -struct pathbuf_entry { -u64 inum; -u32 snapshot; -}; - -typedef DARRAY (struct pathbuf_entry )pathbuf ; - -static int bch2_bi_depth_renumber_one (struct btree_trans *trans,struct pathbuf_entry *p, +static int bch2_bi_depth_renumber_one (struct btree_trans *trans, +u64 inum,u32 snapshot,u32 new_depth) {struct btree_iter iter;struct bkey_s_c k =bch2_bkey_get_iter (trans, &iter, BTREE_ID_inodes, -SPOS (0 , p->inum, p->snapshot),0 ); +SPOS (0 , inum, snapshot),0 );struct bch_inode_unpacked inode; int ret = bkey_err(k) ?:
@@ -2720,14 +2714,15 @@ static int bch2_bi_depth_renumber_one(struct btree_trans *trans, struct pathbuf_return ret; } -static int bch2_bi_depth_renumber (struct btree_trans *trans,pathbuf *path,u32 new_bi_depth) +static int bch2_bi_depth_renumber (struct btree_trans *trans,darray_u64 *path, +u32 snapshot,u32 new_bi_depth) {u32 restart_count = trans->restart_count;int ret =0 ;darray_for_each_reverse (*path, i) { ret =nested_lockrestart_do (trans, -bch2_bi_depth_renumber_one (trans, i, new_bi_depth)); +bch2_bi_depth_renumber_one (trans, *i, snapshot, new_bi_depth));bch_err_fn (trans->c, ret);if (ret)break ;
@@ -2738,26 +2733,19 @@ static int bch2_bi_depth_renumber(struct btree_trans *trans, pathbuf *path, u32return ret ?:trans_was_restarted (trans, restart_count); } -static bool path_is_dup (pathbuf *p,u64 inum,u32 snapshot) -{ -darray_for_each (*p, i) -if (i->inum == inum && - i->snapshot == snapshot) -return true; -return false; -} -static int check_path_loop (struct btree_trans *trans,struct bkey_s_c inode_k) {struct bch_fs *c = trans->c;struct btree_iter inode_iter = {}; -pathbuf path = {}; +darray_u64 path = {};struct printbuf buf =PRINTBUF ;u32 snapshot = inode_k.k->p.snapshot;bool redo_bi_depth = false;u32 min_bi_depth =U32_MAX ;int ret =0 ; +struct bpos start = inode_k.k->p; +struct bch_inode_unpacked inode; ret =bch2_inode_unpack (inode_k, &inode);if (ret)
@@ -2766,9 +2754,9 @@ static int check_path_loop(struct btree_trans *trans, struct bkey_s_c inode_k)while (!inode.bi_subvol) {struct btree_iter dirent_iter;struct bkey_s_c_dirent d; -u32 parent_snapshot = snapshot; - d =inode_get_dirent (trans, &dirent_iter, &inode, &parent_snapshot); + d =dirent_get_by_pos (trans, &dirent_iter, +SPOS (inode.bi_dir, inode.bi_dir_offset, snapshot)); ret =bkey_err (d.s_c);if (ret && !bch2_err_matches (ret,ENOENT )) goto out;
@@ -2786,15 +2774,10 @@ static int check_path_loop(struct btree_trans *trans, struct bkey_s_c inode_k)bch2_trans_iter_exit (trans, &dirent_iter); - ret =darray_push (&path, ((struct pathbuf_entry ) { - .inum = inode.bi_inum, - .snapshot = snapshot, - })); + ret =darray_push (&path, inode.bi_inum);if (ret)return ret; - snapshot = parent_snapshot; -bch2_trans_iter_exit (trans, &inode_iter); inode_k =bch2_bkey_get_iter (trans, &inode_iter, BTREE_ID_inodes,SPOS (0 , inode.bi_dir, snapshot),0 );
@@ -2816,15 +2799,22 @@ static int check_path_loop(struct btree_trans *trans, struct bkey_s_c inode_k)break ; inode = parent_inode; - snapshot = inode_k.k->p.snapshot; redo_bi_depth = true; -if (path_is_dup (&path, inode.bi_inum, snapshot)) { +if (darray_find (path, inode.bi_inum)) {printbuf_reset (&buf); -prt_printf (&buf,"directory structure loop:\n" ); -darray_for_each_reverse (path, i) -prt_printf (&buf,"%llu:%u " , i->inum, i->snapshot); -prt_printf (&buf,"%llu:%u" , inode.bi_inum, snapshot); +prt_printf (&buf,"directory structure loop in snapshot %u: " , + snapshot); + + ret =bch2_inum_snapshot_to_path (trans, start.offset, start.snapshot,NULL , &buf); +if (ret) + goto out; + +if (c->opts.verbose) { +prt_newline (&buf); +darray_for_each (path, i) +prt_printf (&buf,"%llu " , *i); + }if (fsck_err (trans, dir_loop,"%s" , buf.buf)) { ret =remove_backpointer (trans, &inode);
@@ -2844,7 +2834,7 @@ static int check_path_loop(struct btree_trans *trans, struct bkey_s_c inode_k) min_bi_depth =0 ;if (redo_bi_depth) - ret =bch2_bi_depth_renumber (trans, &path, min_bi_depth); + ret =bch2_bi_depth_renumber (trans, &path, snapshot, min_bi_depth); out: fsck_err:bch2_trans_iter_exit (trans, &inode_iter);
Signature · 1 line
2.50.0
PATCH 31/37 bcachefs: Fix bch2_read_bio_to_text()
In reply to Kent OverstreetWe can only pass negative error codes to bch2_err_str(); if it's a positive integer it's not an error and we trip an assert.
--- fs/bcachefs/io_read.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
fs/bcachefs/io_read.c +6 -1
@@ -1491,7 +1491,12 @@ void bch2_read_bio_to_text(struct printbuf *out, struct bch_read_bio *rbio)prt_printf (out,"have_ioref:\t%u\n" , rbio->have_ioref);prt_printf (out,"narrow_crcs:\t%u\n" , rbio->narrow_crcs);prt_printf (out,"context:\t%u\n" , rbio->context); -prt_printf (out,"ret:\t%s\n" ,bch2_err_str (rbio->ret)); + +int ret =READ_ONCE (rbio->ret); +if (ret <0 ) +prt_printf (out,"ret:\t%s\n" ,bch2_err_str (ret)); +else +prt_printf (out,"ret:\t%i\n" , ret);prt_printf (out,"flags:\t" );bch2_prt_bitflags (out, bch2_read_bio_flags, rbio->flags);
Signature · 1 line
2.50.0
PATCH 32/37 bcachefs: Fix restart handling in btree_node_scrub_work()
In reply to Kent Overstreetbtree node scrub was sometimes failing to rewrite nodes with errors; bch2_btree_node_rewrite() can return a transaction restart and we weren't checking - the lockrestart_do() needs to wrap the entire operation.
And there's a better helper it should've been using, bch2_btree_node_rewrite_key(), which makes all this more convenient.
--- fs/bcachefs/btree_io.c | 28 ++++++---------------------- fs/bcachefs/btree_update_interior.c | 11 +++++------ fs/bcachefs/btree_update_interior.h | 3 +++ 3 files changed, 14 insertions(+), 28 deletions(-)
fs/bcachefs/btree_io.c +6 -22
@@ -1986,28 +1986,12 @@ static void btree_node_scrub_work(struct work_struct *work)prt_newline (&err);if (!btree_node_scrub_check (c, scrub->buf, scrub->written, &err)) { -struct btree_trans *trans =bch2_trans_get (c); - -struct btree_iter iter; -bch2_trans_node_iter_init (trans, &iter, scrub->btree, - scrub->key.k->k.p,0 , scrub->level -1 ,0 ); - -struct btree *b; -int ret =lockrestart_do (trans, -PTR_ERR_OR_ZERO (b =bch2_btree_iter_peek_node (trans, &iter))); -if (ret) - goto err; - -if (bkey_i_to_btree_ptr_v2 (&b->key)->v.seq == scrub->seq) { -bch_err (c,"error validating btree node during scrub on %s at btree %s" , - scrub->ca->name, err.buf); - - ret =bch2_btree_node_rewrite (trans, &iter, b,0 ,0 ); - } -err: -bch2_trans_iter_exit (trans, &iter); -bch2_trans_begin (trans); -bch2_trans_put (trans); +int ret =bch2_trans_do (c, +bch2_btree_node_rewrite_key (trans, scrub->btree, scrub->level -1 , + scrub->key.k,0 )); +if (!bch2_err_matches (ret,ENOENT ) && + !bch2_err_matches (ret,EROFS )) +bch_err_fn_ratelimited (c, ret); }printbuf_exit (&err);
fs/bcachefs/btree_update_interior.c +5 -6
@@ -2293,9 +2293,9 @@ int bch2_btree_node_rewrite(struct btree_trans *trans, goto out; } -static int bch2_btree_node_rewrite_key (struct btree_trans *trans, -enum btree_id btree,unsigned level, -struct bkey_i *k,unsigned flags) +int bch2_btree_node_rewrite_key (struct btree_trans *trans, +enum btree_id btree,unsigned level, +struct bkey_i *k,unsigned flags) {struct btree_iter iter; bch2_trans_node_iter_init(trans, &iter,
@@ -2367,9 +2367,8 @@ static void async_btree_node_rewrite_work(struct work_struct *work)int ret =bch2_trans_do (c,bch2_btree_node_rewrite_key (trans, a->btree_id, a->level, a->key.k,0 )); -if (ret != -ENOENT && - !bch2_err_matches (ret,EROFS ) && - ret != -BCH_ERR_journal_shutdown) +if (!bch2_err_matches (ret,ENOENT ) && + !bch2_err_matches (ret,EROFS ))bch_err_fn_ratelimited (c, ret);spin_lock (&c->btree_node_rewrites_lock);
fs/bcachefs/btree_update_interior.h +3 -0
@@ -176,6 +176,9 @@ static inline int bch2_foreground_maybe_merge(struct btree_trans *trans,int bch2_btree_node_rewrite (struct btree_trans *,struct btree_iter *,struct btree *,unsigned ,unsigned ); +int bch2_btree_node_rewrite_key (struct btree_trans *, +enum btree_id ,unsigned , +struct bkey_i *,unsigned );int bch2_btree_node_rewrite_pos (struct btree_trans *,enum btree_id ,unsigned ,struct bpos ,unsigned ,unsigned );
Signature · 1 line
2.50.0
PATCH 33/37 bcachefs: fsck: Fix check_directory_structure when no check_dirents
In reply to Kent Overstreetcheck_directory_structure runs after check_dirents, so it expects that it won't see any inodes with missing backpointers - normally.
But online fsck can't run check_dirents yet, or the user might only be running a specific pass, so we need to be careful that this isn't an error. If an inode is unreachable, that's handled by a separate pass.
Also, add a new 'bch2_inode_has_backpointer()' helper, since we were doing this inconsistently.
--- fs/bcachefs/fsck.c | 18 +++++++++++++----- fs/bcachefs/inode.h | 5 +++++ fs/bcachefs/namei.c | 3 +-- 3 files changed, 19 insertions(+), 7 deletions(-)
fs/bcachefs/fsck.c +13 -5
@@ -327,7 +327,8 @@ static inline bool inode_should_reattach(struct bch_inode_unpacked *inode) (inode->bi_flags & BCH_INODE_has_child_snapshot))return false; -return !inode->bi_dir && !(inode->bi_flags & BCH_INODE_unlinked); +return !bch2_inode_has_backpointer (inode) && + !(inode->bi_flags & BCH_INODE_unlinked); }static int maybe_delete_dirent (struct btree_trans *trans,struct bpos d_pos,u32 snapshot)
@@ -514,7 +515,7 @@ static struct bkey_s_c_dirent dirent_get_by_pos(struct btree_trans *trans,static int remove_backpointer (struct btree_trans *trans,struct bch_inode_unpacked *inode) { -if (!inode->bi_dir) +if (!bch2_inode_has_backpointer (inode))return 0 ;u32 snapshot = inode->bi_snapshot;
@@ -1165,13 +1166,14 @@ static int check_inode(struct btree_trans *trans,if (ret) goto err; -if (u.bi_dir || u.bi_dir_offset) { +if (bch2_inode_has_backpointer (&u)) { ret =check_inode_dirent_inode (trans, &u, &do_update);if (ret) goto err; } - if (fsck_err_on(u.bi_dir && (u.bi_flags & BCH_INODE_unlinked), + if (fsck_err_on(bch2_inode_has_backpointer(&u) && + (u.bi_flags & BCH_INODE_unlinked), trans, inode_unlinked_but_has_dirent, "inode unlinked but has dirent\n%s", (printbuf_reset(&buf),
@@ -2751,7 +2753,13 @@ static int check_path_loop(struct btree_trans *trans, struct bkey_s_c inode_k)if (ret)return ret; -while (!inode.bi_subvol) { +/* +* If we're running full fsck, check_dirents() will have already ran, +* and we shouldn't see any missing backpointers here - otherwise that's +* handled separately, by check_unreachable_inodes +*/ +while (!inode.bi_subvol && +bch2_inode_has_backpointer (&inode)) {struct btree_iter dirent_iter;struct bkey_s_c_dirent d;
fs/bcachefs/inode.h +5 -0
@@ -254,6 +254,11 @@ static inline bool bch2_inode_casefold(struct bch_fs *c, const struct bch_inode_ : c->opts.casefold; } +static inline bool bch2_inode_has_backpointer (const struct bch_inode_unpacked *bi) +{ +return bi->bi_dir || bi->bi_dir_offset; +} +/* i_nlink: */ static inline unsigned nlink_bias (umode_t mode)
fs/bcachefs/namei.c +1 -2
@@ -734,8 +734,7 @@ static int bch2_check_dirent_inode_dirent(struct btree_trans *trans,if (inode_points_to_dirent (target, d))return 0 ; -if (!target->bi_dir && - !target->bi_dir_offset) { +if (!bch2_inode_has_backpointer (target)) { fsck_err_on(S_ISDIR(target->bi_mode), trans, inode_dir_missing_backpointer, "directory with missing backpointer\n%s",
Signature · 1 line
2.50.0
PATCH 34/37 bcachefs: fsck: fix unhandled restart in topology repair
In reply to Kent Overstreet--- fs/bcachefs/btree_gc.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
fs/bcachefs/btree_gc.c +7 -1
@@ -503,8 +503,14 @@ static int bch2_btree_repair_topology_recurse(struct btree_trans *trans, structprt_newline (&buf);bch2_bkey_val_to_text (&buf, c,bkey_i_to_s_c (&b->key)); +/* +* XXX: we're not passing the trans object here because we're not set up +* to handle a transaction restart - this code needs to be rewritten +* when we start doing online topology repair +*/ +bch2_trans_unlock_long (trans);if (mustfix_fsck_err_on (!have_child, - trans, btree_node_topology_interior_node_empty, + c, btree_node_topology_interior_node_empty,"empty interior btree node at %s" , buf.buf)) ret =DROP_THIS_NODE ; err:
Signature · 1 line
2.50.0
PATCH 35/37 bcachefs: fsck: Fix oops in key_visible_in_snapshot()
In reply to Kent OverstreetThe normal fsck code doesn't call key_visible_in_snapshot() with an empty list of snapshot IDs seen (the current snapshot ID will always be on the list), but str_hash_repair_key() -> bch2_get_snapshot_overwrites() can, and that's totally fine as long as we check for it.
--- fs/bcachefs/fsck.c | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-)
fs/bcachefs/fsck.c +2 -11
@@ -728,14 +728,8 @@ static int snapshots_seen_update(struct bch_fs *c, struct snapshots_seen *s,static bool key_visible_in_snapshot (struct bch_fs *c,struct snapshots_seen *seen,u32 id,u32 ancestor) { -ssize_t i; -EBUG_ON (id > ancestor); -/* @ancestor should be the snapshot most recently added to @seen */ -EBUG_ON (ancestor != seen->pos.snapshot); -EBUG_ON (ancestor !=darray_last (seen->ids)); -if (id == ancestor)return true;
@@ -751,11 +745,8 @@ static bool key_visible_in_snapshot(struct bch_fs *c, struct snapshots_seen *see* numerically, since snapshot ID lists are kept sorted, so if we find * an id that's an ancestor of @id we're done: */ - -for (i = seen->ids.nr -2 ; - i >=0 && seen->ids.data[i] >= id; - --i) -if (bch2_snapshot_is_ancestor (c, id, seen->ids.data[i])) +darray_for_each_reverse (seen->ids, i) +if (*i != ancestor &&bch2_snapshot_is_ancestor (c, id, *i))return false;return true;
Signature · 1 line
2.50.0
PATCH 36/37 bcachefs: fix spurious error in read_btree_roots()
In reply to Kent Overstreet--- fs/bcachefs/recovery.c | 1 + 1 file changed, 1 insertion(+)
fs/bcachefs/recovery.c +1 -0
@@ -607,6 +607,7 @@ static int read_btree_roots(struct bch_fs *c) buf.buf, bch2_err_str(ret))) {if (btree_id_is_alloc (i)) r->error =0 ; + ret =0 ; } }
Signature · 1 line
2.50.0
PATCH 37/37 bcachefs: Fix missing newlines before ero
In reply to Kent Overstreet--- fs/bcachefs/data_update.c | 1 + fs/bcachefs/journal_io.c | 5 +++-- fs/bcachefs/recovery.c | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-)
fs/bcachefs/data_update.c +1 -0
@@ -249,6 +249,7 @@ static int data_update_invalid_bkey(struct data_update *m,bch2_bkey_val_to_text (&buf, c, k);prt_str (&buf,"\nnew: " );bch2_bkey_val_to_text (&buf, c,bkey_i_to_s_c (insert)); +prt_newline (&buf);bch2_fs_emergency_read_only2 (c, &buf);
fs/bcachefs/journal_io.c +3 -2
@@ -1716,9 +1716,10 @@ static CLOSURE_CALLBACK(journal_write_done)bch2_log_msg_start (c, &buf);if (err == -BCH_ERR_journal_write_err) -prt_printf (&buf,"unable to write journal to sufficient devices" ); +prt_printf (&buf,"unable to write journal to sufficient devices\n" );else -prt_printf (&buf,"journal write error marking replicas: %s" ,bch2_err_str (err)); +prt_printf (&buf,"journal write error marking replicas: %s\n" , +bch2_err_str (err));bch2_fs_emergency_read_only2 (c, &buf);
fs/bcachefs/recovery.c +1 -1
@@ -1142,7 +1142,7 @@ int bch2_fs_recovery(struct bch_fs *c)struct printbuf buf =PRINTBUF ;bch2_log_msg_start (c, &buf); -prt_printf (&buf,"error in recovery: %s" ,bch2_err_str (ret)); +prt_printf (&buf,"error in recovery: %s\n" ,bch2_err_str (ret));bch2_fs_emergency_read_only2 (c, &buf);bch2_print_str (c,KERN_ERR , buf.buf);
Signature · 1 line
2.50.0