enumarated refcounts, for debugging refcount issues
cover enumarated refcounts, for debugging refcount issues
Not sure we have a list for library code, but this might be of interest to anyone who's had to debug refcount issues on refs with lots of users (filesystem people), and I know the hardening folks deal with refcounts a lot.
In release mode, this is just a wrapper around percpu refcounts.
In debug mode, this provides separate sub-refcounts for each enumerated user, and provides facilities for printing them.
Meaning, if you're debugging a refcount issue, you no longer have to go searching through the entire codebase - it'll tell you the exact codepath.
bcachefs patches are provided as example usage, for other subsystems.
Kent Overstreet (3):
bcachefs: enumerated_ref.c
bcachefs: bch_fs.writes -> enumerated_refs
bcachefs: bch_dev.io_ref -> enumerated_ref
fs/bcachefs/Makefile | 1 + fs/bcachefs/alloc_background.c | 41 ++++---- fs/bcachefs/backpointers.c | 6 +- fs/bcachefs/bcachefs.h | 105 ++++++++++---------- fs/bcachefs/btree_gc.c | 7 +- fs/bcachefs/btree_io.c | 29 +++--- fs/bcachefs/btree_node_scan.c | 10 +- fs/bcachefs/btree_trans_commit.c | 5 +- fs/bcachefs/btree_update_interior.c | 7 +- fs/bcachefs/btree_write_buffer.c | 11 ++- fs/bcachefs/buckets.c | 4 +- fs/bcachefs/debug.c | 12 ++- fs/bcachefs/ec.c | 32 ++++--- fs/bcachefs/enumerated_ref.c | 144 ++++++++++++++++++++++++++++ fs/bcachefs/enumerated_ref.h | 66 +++++++++++++ fs/bcachefs/enumerated_ref_types.h | 19 ++++ fs/bcachefs/fs-io-direct.c | 7 +- fs/bcachefs/fs-io.c | 15 +-- fs/bcachefs/io_read.c | 17 ++-- fs/bcachefs/io_write.c | 20 ++-- fs/bcachefs/journal.c | 10 +- fs/bcachefs/journal_io.c | 15 +-- fs/bcachefs/journal_reclaim.c | 2 +- fs/bcachefs/reflink.c | 5 +- fs/bcachefs/sb-members.h | 32 ++++--- fs/bcachefs/snapshot.c | 7 +- fs/bcachefs/subvolume.c | 7 +- fs/bcachefs/super-io.c | 18 ++-- fs/bcachefs/super.c | 125 +++++++++++------------- fs/bcachefs/super.h | 3 + fs/bcachefs/sysfs.c | 46 ++++----- 31 files changed, 540 insertions(+), 288 deletions(-) create mode 100644 fs/bcachefs/enumerated_ref.c create mode 100644 fs/bcachefs/enumerated_ref.h create mode 100644 fs/bcachefs/enumerated_ref_types.h
Signature · 1 line
2.49.0
PATCH 1/3 bcachefs: enumerated_ref.c
Factor out the debug code for rw filesystem refs into a small library.
In release mode an enumerated ref is a normal percpu refcount, but in debug mode all enumerated users of the ref get their own atomic_long_t ref - making it much easier to chase down refcount usage bugs for when a refcount has many users.
For debugging, we have enumerated_ref_to_text(), which prints the current value of each different user.
Additionally, in debug mode enumerated_ref_stop() has a 10 second timeout, after which it will dump outstanding refcounts.
--- fs/bcachefs/Makefile | 1 + fs/bcachefs/enumerated_ref.c | 144 +++++++++++++++++++++++++++++ fs/bcachefs/enumerated_ref.h | 54 +++++++++++ fs/bcachefs/enumerated_ref_types.h | 19 ++++ 4 files changed, 218 insertions(+) create mode 100644 fs/bcachefs/enumerated_ref.c create mode 100644 fs/bcachefs/enumerated_ref.h create mode 100644 fs/bcachefs/enumerated_ref_types.h
fs/bcachefs/Makefile +1 -0
@@ -35,6 +35,7 @@ bcachefs-y := \ disk_accounting.o \ disk_groups.o \ ec.o \ + enumerated_ref.o \ errcode.o \ error.o \ extents.o \
fs/bcachefs/enumerated_ref.c new file +144 -0
144 lines not on this page · raw message
fs/bcachefs/enumerated_ref.h new file +54 -0
54 lines not on this page · raw message
fs/bcachefs/enumerated_ref_types.h new file +19 -0
@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_ENUMERATED_REF_TYPES_H +#define _BCACHEFS_ENUMERATED_REF_TYPES_H + +#include <linux/percpu-refcount.h> + +struct enumerated_ref { +#ifdef ENUMERATED_REF_DEBUG +unsigned nr; +bool dying; +atomic_long_t *refs; +#else +struct percpu_ref ref; +#endif +void (*stop_fn)(struct enumerated_ref *); +struct completion stop_complete; +}; + +#endif /* _BCACHEFS_ENUMERATED_REF_TYPES_H */
Signature · 1 line
2.49.0
PATCH 2/3 bcachefs: bch_fs.writes -> enumerated_refs
In reply to Kent OverstreetDrop the single-purpose write ref code in bcachefs.h, and convert to enumarated refs.
--- fs/bcachefs/alloc_background.c | 23 ++++++------ fs/bcachefs/bcachefs.h | 57 ++--------------------------- fs/bcachefs/btree_gc.c | 7 ++-- fs/bcachefs/btree_io.c | 7 ++-- fs/bcachefs/btree_trans_commit.c | 5 ++- fs/bcachefs/btree_update_interior.c | 7 ++-- fs/bcachefs/btree_write_buffer.c | 11 +++--- fs/bcachefs/ec.c | 13 ++++--- fs/bcachefs/fs-io-direct.c | 7 ++-- fs/bcachefs/fs-io.c | 9 +++-- fs/bcachefs/io_read.c | 7 ++-- fs/bcachefs/io_write.c | 5 ++- fs/bcachefs/journal.c | 5 ++- fs/bcachefs/reflink.c | 5 ++- fs/bcachefs/snapshot.c | 7 ++-- fs/bcachefs/subvolume.c | 7 ++-- fs/bcachefs/super.c | 32 ++++------------ fs/bcachefs/sysfs.c | 26 +++---------- 18 files changed, 87 insertions(+), 153 deletions(-)
fs/bcachefs/alloc_background.c +12 -11
@@ -17,6 +17,7 @@#include "debug.h" #include "disk_accounting.h" #include "ec.h" +#include "enumerated_ref.h" #include "error.h" #include "lru.h" #include "recovery.h"
@@ -1381,7 +1382,7 @@ static void check_discard_freespace_key_work(struct work_struct *work) container_of(work, struct check_discard_freespace_key_async, work);bch2_trans_do (w->c,bch2_recheck_discard_freespace_key (trans, w->pos)); -bch2_write_ref_put (w->c, BCH_WRITE_REF_check_discard_freespace_key); +enumerated_ref_put (&w->c->writes, BCH_WRITE_REF_check_discard_freespace_key);kfree (w); }
@@ -1458,7 +1459,7 @@ int bch2_check_discard_freespace_key(struct btree_trans *trans, struct btree_iteif (!w) goto out; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_check_discard_freespace_key)) { +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_check_discard_freespace_key)) {kfree (w); goto out; }
@@ -1953,14 +1954,14 @@ static void bch2_do_discards_work(struct work_struct *work) bch2_err_str(ret));percpu_ref_put (&ca->io_ref[WRITE ]); -bch2_write_ref_put (c, BCH_WRITE_REF_discard); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard); }void bch2_dev_do_discards (struct bch_dev *ca) {struct bch_fs *c = ca->fs; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_discard)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_discard))return ;if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE ))
@@ -1971,7 +1972,7 @@ void bch2_dev_do_discards(struct bch_dev *ca)percpu_ref_put (&ca->io_ref[WRITE ]); put_write_ref: -bch2_write_ref_put (c, BCH_WRITE_REF_discard); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard); }void bch2_do_discards (struct bch_fs *c)
@@ -2048,7 +2049,7 @@ static void bch2_do_discards_fast_work(struct work_struct *work)bch2_trans_put (trans);percpu_ref_put (&ca->io_ref[WRITE ]); -bch2_write_ref_put (c, BCH_WRITE_REF_discard_fast); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard_fast); }static void bch2_discard_one_bucket_fast (struct bch_dev *ca,u64 bucket)
@@ -2058,7 +2059,7 @@ static void bch2_discard_one_bucket_fast(struct bch_dev *ca, u64 bucket)if (discard_in_flight_add (ca, bucket, false))return ; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_discard_fast)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_discard_fast))return ;if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE ))
@@ -2069,7 +2070,7 @@ static void bch2_discard_one_bucket_fast(struct bch_dev *ca, u64 bucket)percpu_ref_put (&ca->io_ref[WRITE ]); put_ref: -bch2_write_ref_put (c, BCH_WRITE_REF_discard_fast); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard_fast); } static int invalidate_one_bp(struct btree_trans *trans,
@@ -2263,14 +2264,14 @@ static void bch2_do_invalidates_work(struct work_struct *work)bch2_trans_put (trans);percpu_ref_put (&ca->io_ref[WRITE ]);bch2_bkey_buf_exit (&last_flushed, c); -bch2_write_ref_put (c, BCH_WRITE_REF_invalidate); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_invalidate); }void bch2_dev_do_invalidates (struct bch_dev *ca) {struct bch_fs *c = ca->fs; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_invalidate)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_invalidate))return ;if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE ))
@@ -2281,7 +2282,7 @@ void bch2_dev_do_invalidates(struct bch_dev *ca)percpu_ref_put (&ca->io_ref[WRITE ]); put_ref: -bch2_write_ref_put (c, BCH_WRITE_REF_invalidate); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_invalidate); }void bch2_do_invalidates (struct bch_fs *c)
fs/bcachefs/bcachefs.h +3 -54
81 lines not on this page · raw message
fs/bcachefs/btree_gc.c +4 -3
@@ -22,6 +22,7 @@#include "debug.h" #include "disk_accounting.h" #include "ec.h" +#include "enumerated_ref.h" #include "error.h" #include "extents.h" #include "journal.h"
@@ -1233,14 +1234,14 @@ static void bch2_gc_gens_work(struct work_struct *work) { struct bch_fs *c = container_of(work, struct bch_fs, gc_gens_work);bch2_gc_gens (c); -bch2_write_ref_put (c, BCH_WRITE_REF_gc_gens); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_gc_gens); }void bch2_gc_gens_async (struct bch_fs *c) { -if (bch2_write_ref_tryget (c, BCH_WRITE_REF_gc_gens) && +if (enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_gc_gens) && !queue_work (c->write_ref_wq, &c->gc_gens_work)) -bch2_write_ref_put (c, BCH_WRITE_REF_gc_gens); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_gc_gens); }void bch2_fs_btree_gc_init_early (struct bch_fs *c)
fs/bcachefs/btree_io.c +4 -3
@@ -13,6 +13,7 @@#include "buckets.h" #include "checksum.h" #include "debug.h" +#include "enumerated_ref.h" #include "error.h" #include "extents.h" #include "io_write.h"
@@ -1935,7 +1936,7 @@ static void btree_node_scrub_work(struct work_struct *work)btree_bounce_free (c, c->opts.btree_node_size, scrub->used_mempool, scrub->buf);percpu_ref_put (&scrub->ca->io_ref[READ ]);kfree (scrub); -bch2_write_ref_put (c, BCH_WRITE_REF_btree_node_scrub); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_btree_node_scrub); }static void btree_node_scrub_endio (struct bio *bio)
@@ -1954,7 +1955,7 @@ int bch2_btree_node_scrub(struct btree_trans *trans,struct bch_fs *c = trans->c; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_btree_node_scrub)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_btree_node_scrub))return -BCH_ERR_erofs_no_writes;struct extent_ptr_decoded pick;
@@ -2004,7 +2005,7 @@ int bch2_btree_node_scrub(struct btree_trans *trans,btree_bounce_free (c, c->opts.btree_node_size, used_mempool, buf);percpu_ref_put (&ca->io_ref[READ ]); err: -bch2_write_ref_put (c, BCH_WRITE_REF_btree_node_scrub); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_btree_node_scrub);return ret; }
fs/bcachefs/btree_trans_commit.c +3 -2
@@ -11,6 +11,7 @@#include "btree_write_buffer.h" #include "buckets.h" #include "disk_accounting.h" +#include "enumerated_ref.h" #include "errcode.h" #include "error.h" #include "journal.h"
@@ -994,7 +995,7 @@ int __bch2_trans_commit(struct btree_trans *trans, unsigned flags) goto out_reset;if (!(flags & BCH_TRANS_COMMIT_no_check_rw) && -unlikely (!bch2_write_ref_tryget (c, BCH_WRITE_REF_trans))) { +unlikely (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_trans))) {if (unlikely (!test_bit (BCH_FS_may_go_rw, &c->flags))) ret =do_bch2_trans_commit_to_journal_replay (trans);else
@@ -1060,7 +1061,7 @@ int __bch2_trans_commit(struct btree_trans *trans, unsigned flags)trace_and_count (c, transaction_commit, trans, _RET_IP_); out:if (likely (!(flags & BCH_TRANS_COMMIT_no_check_rw))) -bch2_write_ref_put (c, BCH_WRITE_REF_trans); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_trans); out_reset:if (!ret)bch2_trans_downgrade (trans);
fs/bcachefs/btree_update_interior.c +4 -3
@@ -14,6 +14,7 @@#include "btree_locking.h" #include "buckets.h" #include "clock.h" +#include "enumerated_ref.h" #include "error.h" #include "extents.h" #include "io_write.h"
@@ -2326,7 +2327,7 @@ static void async_btree_node_rewrite_work(struct work_struct *work)closure_wake_up (&c->btree_node_rewrites_wait);bch2_bkey_buf_exit (&a->key, c); -bch2_write_ref_put (c, BCH_WRITE_REF_node_rewrite); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_node_rewrite);kfree (a); }
@@ -2348,7 +2349,7 @@ void bch2_btree_node_rewrite_async(struct bch_fs *c, struct btree *b)spin_lock (&c->btree_node_rewrites_lock);if (c->curr_recovery_pass > BCH_RECOVERY_PASS_journal_replay && -bch2_write_ref_tryget (c, BCH_WRITE_REF_node_rewrite)) { +enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_node_rewrite)) {list_add (&a->list, &c->btree_node_rewrites); now = true; }else if (!test_bit (BCH_FS_may_go_rw, &c->flags)) {
@@ -2387,7 +2388,7 @@ void bch2_do_pending_node_rewrites(struct bch_fs *c)if (!a)break ; -bch2_write_ref_get (c, BCH_WRITE_REF_node_rewrite); +enumerated_ref_get (&c->writes, BCH_WRITE_REF_node_rewrite);queue_work (c->btree_node_rewrite_worker, &a->work); } }
fs/bcachefs/btree_write_buffer.c +6 -5
@@ -7,6 +7,7 @@#include "btree_update_interior.h" #include "btree_write_buffer.h" #include "disk_accounting.h" +#include "enumerated_ref.h" #include "error.h" #include "extents.h" #include "journal.h"
@@ -629,11 +630,11 @@ int bch2_btree_write_buffer_tryflush(struct btree_trans *trans) {struct bch_fs *c = trans->c; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_btree_write_buffer)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_btree_write_buffer))return -BCH_ERR_erofs_no_writes;int ret =bch2_btree_write_buffer_flush_nocheck_rw (trans); -bch2_write_ref_put (c, BCH_WRITE_REF_btree_write_buffer); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_btree_write_buffer);return ret; }
@@ -692,7 +693,7 @@ static void bch2_btree_write_buffer_flush_work(struct work_struct *work) }while (!ret &&bch2_btree_write_buffer_should_flush (c));mutex_unlock (&wb->flushing.lock); -bch2_write_ref_put (c, BCH_WRITE_REF_btree_write_buffer); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_btree_write_buffer); }static void wb_accounting_sort (struct btree_write_buffer *wb)
@@ -821,9 +822,9 @@ int bch2_journal_keys_to_write_buffer_end(struct bch_fs *c, struct journal_keys_bch2_journal_pin_drop (&c->journal, &dst->wb->pin);if (bch2_btree_write_buffer_should_flush (c) && -__bch2_write_ref_tryget (c, BCH_WRITE_REF_btree_write_buffer) && +__enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_btree_write_buffer) && !queue_work (system_unbound_wq, &c->btree_write_buffer.flush_work)) -bch2_write_ref_put (c, BCH_WRITE_REF_btree_write_buffer); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_btree_write_buffer);if (dst->wb == &wb->flushing)mutex_unlock (&wb->flushing.lock);
fs/bcachefs/ec.c +7 -6
@@ -16,6 +16,7 @@#include "disk_accounting.h" #include "disk_groups.h" #include "ec.h" +#include "enumerated_ref.h" #include "error.h" #include "io_read.h" #include "io_write.h"
@@ -1017,14 +1018,14 @@ static void ec_stripe_delete_work(struct work_struct *work) BCH_TRANS_COMMIT_no_enospc, ({ec_stripe_delete (trans, lru_k.k->p.offset); }))); -bch2_write_ref_put (c, BCH_WRITE_REF_stripe_delete); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_stripe_delete); }void bch2_do_stripe_deletes (struct bch_fs *c) { -if (bch2_write_ref_tryget (c, BCH_WRITE_REF_stripe_delete) && +if (enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_stripe_delete) && !queue_work (c->write_ref_wq, &c->ec_stripe_delete_work)) -bch2_write_ref_put (c, BCH_WRITE_REF_stripe_delete); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_stripe_delete); }/* stripe creation: */
@@ -1418,15 +1419,15 @@ static void ec_stripe_create_work(struct work_struct *work)while ((s =get_pending_stripe (c)))ec_stripe_create (s); -bch2_write_ref_put (c, BCH_WRITE_REF_stripe_create); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_stripe_create); }void bch2_ec_do_stripe_creates (struct bch_fs *c) { -bch2_write_ref_get (c, BCH_WRITE_REF_stripe_create); +enumerated_ref_get (&c->writes, BCH_WRITE_REF_stripe_create);if (!queue_work (system_long_wq, &c->ec_stripe_create_work)) -bch2_write_ref_put (c, BCH_WRITE_REF_stripe_create); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_stripe_create); }static void ec_stripe_new_set_pending (struct bch_fs *c,struct ec_stripe_head *h)
fs/bcachefs/fs-io-direct.c +4 -3
@@ -3,6 +3,7 @@#include "bcachefs.h" #include "alloc_foreground.h" +#include "enumerated_ref.h" #include "fs.h" #include "fs-io.h" #include "fs-io-direct.h"
@@ -401,7 +402,7 @@ static __always_inline long bch2_dio_write_done(struct dio_write *dio) ret = dio->op.error ?: ((long ) dio->written <<9 );bio_put (&dio->op.wbio.bio); -bch2_write_ref_put (c, BCH_WRITE_REF_dio_write); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_dio_write);/* inode->i_dio_count is our ref on inode and thus bch_fs */ inode_dio_end (&inode->v);
@@ -606,7 +607,7 @@ ssize_t bch2_direct_write(struct kiocb *req, struct iov_iter *iter)prefetch (&inode->ei_inode);prefetch ((void *) &inode->ei_inode +64 ); -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_dio_write)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_dio_write))return -EROFS ;inode_lock (&inode->v);
@@ -675,7 +676,7 @@ ssize_t bch2_direct_write(struct kiocb *req, struct iov_iter *iter)bio_put (bio);inode_dio_end (&inode->v); err_put_write_ref: -bch2_write_ref_put (c, BCH_WRITE_REF_dio_write); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_dio_write); goto out; }
fs/bcachefs/fs-io.c +5 -4
@@ -7,6 +7,7 @@#include "btree_update.h" #include "buckets.h" #include "clock.h" +#include "enumerated_ref.h" #include "error.h" #include "extents.h" #include "extent_update.h"
@@ -205,7 +206,7 @@ static int bch2_flush_inode(struct bch_fs *c,if (c->opts.journal_flush_disabled)return 0 ; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_fsync)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_fsync))return -EROFS ;u64 seq;
@@ -213,7 +214,7 @@ static int bch2_flush_inode(struct bch_fs *c, bch2_get_inode_journal_seq_trans(trans, inode_inum(inode), &seq)) ?:bch2_journal_flush_seq (&c->journal, seq,TASK_INTERRUPTIBLE ) ?:bch2_inode_flush_nocow_writes (c, inode); -bch2_write_ref_put (c, BCH_WRITE_REF_fsync); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_fsync);return ret; }
@@ -796,7 +797,7 @@ long bch2_fallocate_dispatch(struct file *file, int mode,struct bch_fs *c = inode->v.i_sb->s_fs_info;long ret; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_fallocate)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_fallocate))return -EROFS ;inode_lock (&inode->v);
@@ -820,7 +821,7 @@ long bch2_fallocate_dispatch(struct file *file, int mode, err:bch2_pagecache_block_put (inode);inode_unlock (&inode->v); -bch2_write_ref_put (c, BCH_WRITE_REF_fallocate); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_fallocate);return bch2_err_class (ret); }
fs/bcachefs/io_read.c +4 -3
@@ -17,6 +17,7 @@#include "data_update.h" #include "disk_groups.h" #include "ec.h" +#include "enumerated_ref.h" #include "error.h" #include "io_read.h" #include "io_misc.h"
@@ -162,7 +163,7 @@ static noinline void promote_free(struct bch_read_bio *rbio)bch2_data_update_exit (&op->write); -bch2_write_ref_put (c, BCH_WRITE_REF_promote); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_promote);kfree_rcu (op, rcu); }
@@ -227,7 +228,7 @@ static struct bch_read_bio *__promote_alloc(struct btree_trans *trans,return NULL ; } -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_promote)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_promote))return ERR_PTR (-BCH_ERR_nopromote_no_writes);struct promote_op *op =kzalloc (sizeof (*op),GFP_KERNEL );
@@ -278,7 +279,7 @@ static struct bch_read_bio *__promote_alloc(struct btree_trans *trans,/* We may have added to the rhashtable and thus need rcu freeing: */ kfree_rcu (op, rcu); err_put: -bch2_write_ref_put (c, BCH_WRITE_REF_promote); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_promote);return ERR_PTR (ret); }
fs/bcachefs/io_write.c +3 -2
@@ -15,6 +15,7 @@#include "compress.h" #include "debug.h" #include "ec.h" +#include "enumerated_ref.h" #include "error.h" #include "extent_update.h" #include "inode.h"
@@ -512,7 +513,7 @@ static void bch2_write_done(struct closure *cl)bch2_disk_reservation_put (c, &op->res);if (!(op->flags & BCH_WRITE_move)) -bch2_write_ref_put (c, BCH_WRITE_REF_write); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_write);bch2_keylist_free (&op->insert_keys, op->inline_keys);EBUG_ON (cl->parent);
@@ -1660,7 +1661,7 @@ CLOSURE_CALLBACK(bch2_write) }if (!(op->flags & BCH_WRITE_move) && - !bch2_write_ref_tryget (c, BCH_WRITE_REF_write)) { + !enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_write)) { op->error = -BCH_ERR_erofs_no_writes; goto err; }
fs/bcachefs/journal.c +3 -2
@@ -12,6 +12,7 @@#include "btree_update.h" #include "btree_write_buffer.h" #include "buckets.h" +#include "enumerated_ref.h" #include "error.h" #include "journal.h" #include "journal_io.h"
@@ -989,11 +990,11 @@ int bch2_journal_meta(struct journal *j) { struct bch_fs *c = container_of(j, struct bch_fs, journal); -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_journal)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_journal))return -BCH_ERR_erofs_no_writes;int ret =__bch2_journal_meta (j); -bch2_write_ref_put (c, BCH_WRITE_REF_journal); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_journal);return ret; }
fs/bcachefs/reflink.c +3 -2
@@ -3,6 +3,7 @@#include "bkey_buf.h" #include "btree_update.h" #include "buckets.h" +#include "enumerated_ref.h" #include "error.h" #include "extents.h" #include "inode.h"
@@ -610,7 +611,7 @@ s64 bch2_remap_range(struct bch_fs *c, !bch2_request_incompat_feature (c, bcachefs_metadata_version_reflink_p_may_update_opts);int ret =0 , ret2 =0 ; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_reflink)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_reflink))return -BCH_ERR_erofs_no_writes;bch2_check_set_feature (c, BCH_FEATURE_reflink);
@@ -761,7 +762,7 @@ s64 bch2_remap_range(struct bch_fs *c,bch2_bkey_buf_exit (&new_src, c);bch2_bkey_buf_exit (&new_dst, c); -bch2_write_ref_put (c, BCH_WRITE_REF_reflink); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_reflink);return dst_done ?: ret ?: ret2; }
fs/bcachefs/snapshot.c +4 -3
@@ -6,6 +6,7 @@#include "btree_key_cache.h" #include "btree_update.h" #include "buckets.h" +#include "enumerated_ref.h" #include "errcode.h" #include "error.h" #include "fs.h"
@@ -1661,18 +1662,18 @@ void bch2_delete_dead_snapshots_work(struct work_struct *work)set_worker_desc ("bcachefs-delete-dead-snapshots/%s" , c->name);bch2_delete_dead_snapshots (c); -bch2_write_ref_put (c, BCH_WRITE_REF_delete_dead_snapshots); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_delete_dead_snapshots); }void bch2_delete_dead_snapshots_async (struct bch_fs *c) { -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_delete_dead_snapshots)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_delete_dead_snapshots))return ;BUG_ON (!test_bit (BCH_FS_may_go_rw, &c->flags));if (!queue_work (c->write_ref_wq, &c->snapshot_delete_work)) -bch2_write_ref_put (c, BCH_WRITE_REF_delete_dead_snapshots); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_delete_dead_snapshots); } int __bch2_key_has_snapshot_overwrites(struct btree_trans *trans,
fs/bcachefs/subvolume.c +4 -3
@@ -3,6 +3,7 @@#include "bcachefs.h" #include "btree_key_cache.h" #include "btree_update.h" +#include "enumerated_ref.h" #include "errcode.h" #include "error.h" #include "fs.h"
@@ -517,7 +518,7 @@ static void bch2_subvolume_wait_for_pagecache_and_delete(struct work_struct *wordarray_exit (&s); } -bch2_write_ref_put (c, BCH_WRITE_REF_snapshot_delete_pagecache); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_snapshot_delete_pagecache); }struct subvolume_unlink_hook {
@@ -540,11 +541,11 @@ static int bch2_subvolume_wait_for_pagecache_and_delete_hook(struct btree_transif (ret)return ret; -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_snapshot_delete_pagecache)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_snapshot_delete_pagecache))return -EROFS ;if (!queue_work (c->write_ref_wq, &c->snapshot_wait_for_pagecache_and_delete_work)) -bch2_write_ref_put (c, BCH_WRITE_REF_snapshot_delete_pagecache); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_snapshot_delete_pagecache);return 0 ; }
fs/bcachefs/super.c +8 -24
@@ -28,6 +28,7 @@#include "disk_accounting.h" #include "disk_groups.h" #include "ec.h" +#include "enumerated_ref.h" #include "errcode.h" #include "error.h" #include "fs.h"
@@ -311,15 +312,13 @@ static void __bch2_fs_read_only(struct bch_fs *c) } } -#ifndef BCH_WRITE_REF_DEBUG -static void bch2_writes_disabled (struct percpu_ref *writes) +static void bch2_writes_disabled (struct enumerated_ref *writes) { struct bch_fs *c = container_of(writes, struct bch_fs, writes);set_bit (BCH_FS_write_disable_complete, &c->flags);wake_up (&bch2_read_only_wait); } -#endif void bch2_fs_read_only (struct bch_fs *c) {
@@ -337,12 +336,7 @@ void bch2_fs_read_only(struct bch_fs *c)* writes will return -EROFS: */ set_bit (BCH_FS_going_ro, &c->flags); -#ifndef BCH_WRITE_REF_DEBUG -percpu_ref_kill (&c->writes); -#else -for (unsigned i =0 ; i <BCH_WRITE_REF_NR ; i++) -bch2_write_ref_put (c, i); -#endif +enumerated_ref_stop_async (&c->writes);/* * If we're not doing an emergency shutdown, we want to wait on
@@ -525,14 +519,8 @@ static int __bch2_fs_read_write(struct bch_fs *c, bool early)set_bit (BCH_FS_rw, &c->flags);set_bit (BCH_FS_was_rw, &c->flags); -#ifndef BCH_WRITE_REF_DEBUG -percpu_ref_reinit (&c->writes); -#else -for (unsigned i =0 ; i <BCH_WRITE_REF_NR ; i++) { -BUG_ON (atomic_long_read (&c->writes[i])); -atomic_long_inc (&c->writes[i]); - } -#endif +enumerated_ref_start (&c->writes); +if (!early) { ret =bch2_fs_read_write_late (c);if (ret)
@@ -629,9 +617,7 @@ static void __bch2_fs_free(struct bch_fs *c)mempool_exit (&c->btree_bounce_pool);bioset_exit (&c->btree_bio);mempool_exit (&c->fill_iter); -#ifndef BCH_WRITE_REF_DEBUG -percpu_ref_exit (&c->writes); -#endif +enumerated_ref_exit (&c->writes);kfree (rcu_dereference_protected (c->disk_groups,1 ));kfree (c->journal_seq_blacklist_table);
@@ -978,10 +964,8 @@ static struct bch_fs *bch2_fs_alloc(struct bch_sb *sb, struct bch_opts opts,if (!(c->btree_read_complete_wq =alloc_workqueue ("bcachefs_btree_read_complete" ,WQ_HIGHPRI |WQ_FREEZABLE |WQ_MEM_RECLAIM ,512 )) || -#ifndef BCH_WRITE_REF_DEBUG -percpu_ref_init (&c->writes, bch2_writes_disabled, -PERCPU_REF_INIT_DEAD ,GFP_KERNEL ) || -#endif +enumerated_ref_init (&c->writes,BCH_WRITE_REF_NR , + bch2_writes_disabled) ||mempool_init_kmalloc_pool (&c->fill_iter,1 , iter_size) || bioset_init(&c->btree_bio, 1, max(offsetof(struct btree_read_bio, bio),
fs/bcachefs/sysfs.c +6 -20
@@ -25,6 +25,7 @@#include "disk_accounting.h" #include "disk_groups.h" #include "ec.h" +#include "enumerated_ref.h" #include "inode.h" #include "journal.h" #include "journal_reclaim.h"
@@ -177,8 +178,6 @@ read_attribute(btree_reserve_cache);read_attribute (open_buckets);read_attribute (open_buckets_partial);read_attribute (nocow_lock_table); - -#ifdef BCH_WRITE_REF_DEBUG read_attribute (write_refs);static const char *const bch2_write_refs[] = {
@@ -188,15 +187,6 @@ static const char * const bch2_write_refs[] = { NULL }; -static void bch2_write_refs_to_text (struct printbuf *out,struct bch_fs *c) -{ -bch2_printbuf_tabstop_push (out,24 ); - -for (unsigned i =0 ; i <ARRAY_SIZE (c->writes); i++) -prt_printf (out,"%s\t%li\n" , bch2_write_refs[i],atomic_long_read (&c->writes[i])); -} -#endif -read_attribute (internal_uuid);read_attribute (disk_groups);
@@ -481,10 +471,8 @@ SHOW(bch2_fs)if (attr == &sysfs_moving_ctxts)bch2_fs_moving_ctxts_to_text (out, c); -#ifdef BCH_WRITE_REF_DEBUG if (attr == &sysfs_write_refs) -bch2_write_refs_to_text (out, c); -#endif +enumerated_ref_to_text (out, &c->writes, bch2_write_refs);if (attr == &sysfs_nocow_lock_table)bch2_nocow_locks_to_text (out, &c->nocow_locks);
@@ -517,7 +505,7 @@ STORE(bch2_fs)if (attr == &sysfs_trigger_btree_updates)queue_work (c->btree_interior_update_worker, &c->btree_interior_update_work); -if (!bch2_write_ref_tryget (c, BCH_WRITE_REF_sysfs)) +if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_sysfs))return -EROFS ;if (attr == &sysfs_trigger_btree_cache_shrink) {
@@ -577,7 +565,7 @@ STORE(bch2_fs) size = ret; }#endif -bch2_write_ref_put (c, BCH_WRITE_REF_sysfs); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_sysfs);return size; }SYSFS_OPS (bch2_fs);
@@ -670,9 +658,7 @@ struct attribute *bch2_fs_internal_files[] = { &sysfs_new_stripes, &sysfs_open_buckets, &sysfs_open_buckets_partial, -#ifdef BCH_WRITE_REF_DEBUG &sysfs_write_refs, -#endif &sysfs_nocow_lock_table, &sysfs_io_timers_read, &sysfs_io_timers_write,
@@ -738,7 +724,7 @@ static ssize_t sysfs_opt_store(struct bch_fs *c,* We don't need to take c->writes for correctness, but it eliminates an * unsightly error message in the dmesg log when we're RO: */ -if (unlikely (!bch2_write_ref_tryget (c, BCH_WRITE_REF_sysfs))) +if (unlikely (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_sysfs)))return -EROFS ;char *tmp =kstrdup (buf,GFP_KERNEL );
@@ -765,7 +751,7 @@ static ssize_t sysfs_opt_store(struct bch_fs *c, ret = size; err: -bch2_write_ref_put (c, BCH_WRITE_REF_sysfs); +enumerated_ref_put (&c->writes, BCH_WRITE_REF_sysfs);return ret; }
Signature · 1 line
2.49.0
PATCH 3/3 bcachefs: bch_dev.io_ref -> enumerated_ref
In reply to Kent OverstreetConvert device IO refs to enumerated_refs, for easier debugging of refcount issues.
Simple conversion: enumerate all users and convert to the new helpers.
--- fs/bcachefs/alloc_background.c | 18 +++---- fs/bcachefs/backpointers.c | 6 ++- fs/bcachefs/bcachefs.h | 48 +++++++++++++++++- fs/bcachefs/btree_io.c | 22 ++++---- fs/bcachefs/btree_node_scan.c | 10 ++-- fs/bcachefs/buckets.c | 4 +- fs/bcachefs/debug.c | 12 +++-- fs/bcachefs/ec.c | 19 ++++--- fs/bcachefs/enumerated_ref.h | 12 +++++ fs/bcachefs/fs-io.c | 6 ++- fs/bcachefs/io_read.c | 10 ++-- fs/bcachefs/io_write.c | 15 ++++-- fs/bcachefs/journal.c | 5 +- fs/bcachefs/journal_io.c | 15 +++--- fs/bcachefs/journal_reclaim.c | 2 +- fs/bcachefs/sb-members.h | 32 ++++++------ fs/bcachefs/super-io.c | 18 +++---- fs/bcachefs/super.c | 93 ++++++++++++++++++---------------- fs/bcachefs/super.h | 3 ++ fs/bcachefs/sysfs.c | 22 ++++---- 20 files changed, 236 insertions(+), 136 deletions(-)
fs/bcachefs/alloc_background.c +9 -9
@@ -1953,7 +1953,7 @@ static void bch2_do_discards_work(struct work_struct *work)trace_discard_buckets (c, s.seen, s.open, s.need_journal_commit, s.discarded,bch2_err_str (ret)); -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_dev_do_discards);enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard); }
@@ -1964,13 +1964,13 @@ void bch2_dev_do_discards(struct bch_dev *ca)if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_discard))return ; -if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE )) +if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE , BCH_DEV_WRITE_REF_dev_do_discards)) goto put_write_ref;if (queue_work (c->write_ref_wq, &ca->discard_work))return ; -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_dev_do_discards); put_write_ref:enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard); }
@@ -2048,7 +2048,7 @@ static void bch2_do_discards_fast_work(struct work_struct *work)trace_discard_buckets_fast (c, s.seen, s.open, s.need_journal_commit, s.discarded,bch2_err_str (ret));bch2_trans_put (trans); -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_discard_one_bucket_fast);enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard_fast); }
@@ -2062,13 +2062,13 @@ static void bch2_discard_one_bucket_fast(struct bch_dev *ca, u64 bucket)if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_discard_fast))return ; -if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE )) +if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE , BCH_DEV_WRITE_REF_discard_one_bucket_fast)) goto put_ref;if (queue_work (c->write_ref_wq, &ca->discard_fast_work))return ; -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_discard_one_bucket_fast); put_ref:enumerated_ref_put (&c->writes, BCH_WRITE_REF_discard_fast); }
@@ -2262,8 +2262,8 @@ static void bch2_do_invalidates_work(struct work_struct *work)bch2_trans_iter_exit (trans, &iter); err:bch2_trans_put (trans); -percpu_ref_put (&ca->io_ref[WRITE ]);bch2_bkey_buf_exit (&last_flushed, c); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_do_invalidates);enumerated_ref_put (&c->writes, BCH_WRITE_REF_invalidate); }
@@ -2274,13 +2274,13 @@ void bch2_dev_do_invalidates(struct bch_dev *ca)if (!enumerated_ref_tryget (&c->writes, BCH_WRITE_REF_invalidate))return ; -if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE )) +if (!bch2_dev_get_ioref (c, ca->dev_idx,WRITE , BCH_DEV_WRITE_REF_do_invalidates)) goto put_ref;if (queue_work (c->write_ref_wq, &ca->invalidate_work))return ; -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_do_invalidates); put_ref:enumerated_ref_put (&c->writes, BCH_WRITE_REF_invalidate); }
fs/bcachefs/backpointers.c +4 -2
@@ -437,7 +437,8 @@ static int check_extent_checksum(struct btree_trans *trans, bytes = p.crc.compressed_size <<9 ; -struct bch_dev *ca =bch2_dev_get_ioref (c, dev,READ ); +struct bch_dev *ca =bch2_dev_get_ioref (c, dev,READ , + BCH_DEV_READ_REF_check_extent_checksums);if (!ca)return false;
@@ -474,7 +475,8 @@ static int check_extent_checksum(struct btree_trans *trans,if (bio)bio_put (bio);kvfree (data_buf); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], + BCH_DEV_READ_REF_check_extent_checksums);printbuf_exit (&buf);return ret; }
fs/bcachefs/bcachefs.h +46 -2
60 lines not on this page · raw message
fs/bcachefs/btree_io.c +13 -9
@@ -1325,7 +1325,7 @@ static void btree_node_read_work(struct work_struct *work)while (1 ) { retry = true;bch_info (c,"retrying read" ); - ca =bch2_dev_get_ioref (c, rb->pick.ptr.dev,READ ); + ca =bch2_dev_get_ioref (c, rb->pick.ptr.dev,READ , BCH_DEV_READ_REF_btree_node_read); rb->have_ioref = ca !=NULL ; rb->start_time =local_clock ();bio_reset (bio,NULL ,REQ_OP_READ |REQ_SYNC |REQ_META );
@@ -1350,7 +1350,7 @@ static void btree_node_read_work(struct work_struct *work)"btree read error %s for %s" , bch2_blk_status_to_str(bio->bi_status), buf.buf);if (rb->have_ioref) -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_btree_node_read); rb->have_ioref = false;bch2_mark_io_failure (&failed, &rb->pick, false);
@@ -1612,7 +1612,8 @@ static void btree_node_read_all_replicas_endio(struct bio *bio)struct bch_dev *ca =bch2_dev_have_ref (c, rb->pick.ptr.dev);bch2_latency_acct (ca, rb->start_time,READ ); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], + BCH_DEV_READ_REF_btree_node_read_all_replicas); } ra->err[rb->idx] = bio->bi_status;
@@ -1652,7 +1653,8 @@ static int btree_node_read_all_replicas(struct bch_fs *c, struct btree *b, bool i =0 ;bkey_for_each_ptr_decode (k.k, ptrs, pick, entry) { -struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ ); +struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ , + BCH_DEV_READ_REF_btree_node_read_all_replicas);struct btree_read_bio *rb = container_of(ra->bio[i], struct btree_read_bio, bio); rb->c = c;
@@ -1729,7 +1731,7 @@ void bch2_btree_node_read(struct btree_trans *trans, struct btree *b,return ; } - ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ ); + ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ , BCH_DEV_READ_REF_btree_node_read); bio = bio_alloc_bioset(NULL, buf_pages(b->data, btree_buf_bytes(b)),
@@ -1934,7 +1936,7 @@ static void btree_node_scrub_work(struct work_struct *work)printbuf_exit (&err);bch2_bkey_buf_exit (&scrub->key, c);;btree_bounce_free (c, c->opts.btree_node_size, scrub->used_mempool, scrub->buf); -percpu_ref_put (&scrub->ca->io_ref[READ ]); +enumerated_ref_put (&scrub->ca->io_ref[READ ], BCH_DEV_READ_REF_btree_node_scrub);kfree (scrub);enumerated_ref_put (&c->writes, BCH_WRITE_REF_btree_node_scrub); }
@@ -1963,7 +1965,8 @@ int bch2_btree_node_scrub(struct btree_trans *trans,if (ret <=0 ) goto err; -struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ ); +struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ , + BCH_DEV_READ_REF_btree_node_scrub);if (!ca) { ret = -BCH_ERR_device_offline; goto err;
@@ -2003,7 +2006,7 @@ int bch2_btree_node_scrub(struct btree_trans *trans,return 0 ; err_free:btree_bounce_free (c, c->opts.btree_node_size, used_mempool, buf); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_btree_node_scrub); err:enumerated_ref_put (&c->writes, BCH_WRITE_REF_btree_node_scrub);return ret;
@@ -2172,7 +2175,8 @@ static void btree_node_write_endio(struct bio *bio)* btree writes yet (due to device removal/ro): */ if (wbio->have_ioref) -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], + BCH_DEV_READ_REF_btree_node_write);if (parent) {bio_put (bio);
fs/bcachefs/btree_node_scan.c +5 -5
@@ -271,7 +271,7 @@ static int read_btree_nodes_worker(void *p) err:bio_put (bio);free_page ((unsigned long ) buf); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_btree_node_scan);closure_put (w->cl);kfree (w);return 0 ;
@@ -285,13 +285,13 @@ static int read_btree_nodes(struct find_btree_nodes *f)closure_init_stack (&cl); -for_each_online_member (c, ca) { +for_each_online_member (c, ca, BCH_DEV_READ_REF_btree_node_scan) {if (!(ca->mi.data_allowed &BIT (BCH_DATA_btree)))continue ;struct find_btree_nodes_worker *w =kmalloc (sizeof (*w),GFP_KERNEL );if (!w) { -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_btree_node_scan); ret = -ENOMEM ; goto err; }
@@ -303,14 +303,14 @@ static int read_btree_nodes(struct find_btree_nodes *f)struct task_struct *t =kthread_create (read_btree_nodes_worker, w,"read_btree_nodes/%s" , ca->name); ret =PTR_ERR_OR_ZERO (t);if (ret) { -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_btree_node_scan);kfree (w);bch_err_msg (c, ret,"starting kthread" );break ; }closure_get (&cl); -percpu_ref_get (&ca->io_ref[READ ]); +enumerated_ref_get (&ca->io_ref[READ ], BCH_DEV_READ_REF_btree_node_scan);wake_up_process (t); } err:
fs/bcachefs/buckets.c +2 -2
@@ -1139,10 +1139,10 @@ int bch2_trans_mark_dev_sb(struct bch_fs *c, struct bch_dev *ca,int bch2_trans_mark_dev_sbs_flags (struct bch_fs *c,enum btree_iter_update_trigger_flags flags) { -for_each_online_member (c, ca) { +for_each_online_member (c, ca, BCH_DEV_READ_REF_trans_mark_dev_sbs) {int ret =bch2_trans_mark_dev_sb (c, ca, flags);if (ret) { -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_trans_mark_dev_sbs);return ret; } }
fs/bcachefs/debug.c +8 -4
@@ -43,7 +43,8 @@ static bool bch2_btree_verify_replica(struct bch_fs *c, struct btree *b,struct bio *bio;bool failed = false, saw_error = false; -struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ ); +struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ , + BCH_DEV_READ_REF_btree_verify_replicas);if (!ca)return false;
@@ -58,7 +59,8 @@ static bool bch2_btree_verify_replica(struct bch_fs *c, struct btree *b,submit_bio_wait (bio);bio_put (bio); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], + BCH_DEV_READ_REF_btree_verify_replicas);memcpy (n_ondisk, n_sorted,btree_buf_bytes (b));
@@ -197,7 +199,8 @@ void bch2_btree_node_ondisk_to_text(struct printbuf *out, struct bch_fs *c,return ; } - ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ ); + ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ , + BCH_DEV_READ_REF_btree_node_ondisk_to_text);if (!ca) {prt_printf (out,"error getting device to read from: not online\n" );return ;
@@ -298,7 +301,8 @@ void bch2_btree_node_ondisk_to_text(struct printbuf *out, struct bch_fs *c,if (bio)bio_put (bio);kvfree (n_ondisk); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], + BCH_DEV_READ_REF_btree_node_ondisk_to_text); }#ifdef CONFIG_DEBUG_FS
fs/bcachefs/ec.c +13 -6
@@ -707,6 +707,9 @@ static void ec_block_endio(struct bio *bio)struct bch_dev *ca = ec_bio->ca;struct closure *cl = bio->bi_private;int rw = ec_bio->rw; +unsigned ref = rw ==READ + ? BCH_DEV_READ_REF_ec_block + : BCH_DEV_WRITE_REF_ec_block;bch2_account_io_completion (ca,bio_data_dir (bio), ec_bio->submit_time, !bio->bi_status);
@@ -728,7 +731,7 @@ static void ec_block_endio(struct bio *bio) }bio_put (&ec_bio->bio); -percpu_ref_put (&ca->io_ref[rw]); +enumerated_ref_put (&ca->io_ref[rw], ref);closure_put (cl); }
@@ -742,8 +745,11 @@ static void ec_block_io(struct bch_fs *c, struct ec_stripe_buf *buf, ? BCH_DATA_user : BCH_DATA_parity;int rw =op_is_write (opf); +unsigned ref = rw ==READ + ? BCH_DEV_READ_REF_ec_block + : BCH_DEV_WRITE_REF_ec_block; -struct bch_dev *ca =bch2_dev_get_ioref (c, ptr->dev, rw); +struct bch_dev *ca =bch2_dev_get_ioref (c, ptr->dev, rw, ref);if (!ca) {clear_bit (idx, buf->valid);return ;
@@ -789,14 +795,14 @@ static void ec_block_io(struct bch_fs *c, struct ec_stripe_buf *buf,bch2_bio_map (&ec_bio->bio, buf->data[idx] + offset, b);closure_get (cl); -percpu_ref_get (&ca->io_ref[rw]); +enumerated_ref_get (&ca->io_ref[rw], ref);submit_bio (&ec_bio->bio); offset += b; } -percpu_ref_put (&ca->io_ref[rw]); +enumerated_ref_put (&ca->io_ref[rw], ref); } static int get_stripe_key_trans(struct btree_trans *trans, u64 idx,
@@ -1253,7 +1259,8 @@ static void zero_out_rest_of_ec_bucket(struct bch_fs *c,unsigned block, struct open_bucket *ob) { - struct bch_dev *ca = bch2_dev_get_ioref(c, ob->dev, WRITE); + struct bch_dev *ca = bch2_dev_get_ioref(c, ob->dev, WRITE, + BCH_DEV_WRITE_REF_ec_bucket_zero);if (!ca) { s->err = -BCH_ERR_erofs_no_writes;return ;
@@ -1269,7 +1276,7 @@ static void zero_out_rest_of_ec_bucket(struct bch_fs *c, ob->sectors_free, GFP_KERNEL, 0); -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_ec_bucket_zero);if (ret) s->err = ret;
fs/bcachefs/enumerated_ref.h +12 -0
@@ -38,6 +38,18 @@ static inline void enumerated_ref_put(struct enumerated_ref *ref, unsigned idx) }#endif +static inline bool enumerated_ref_is_zero (struct enumerated_ref *ref) +{ +#ifndef BCH_REFCOUNT_DEBUG +return percpu_ref_is_zero (&ref->ref); +#else +for (unsigned i =0 ; i < ref->nr; i++) +if (atomic_long_read (&ref->refs[i])) +return false; +return true; +#endif +} +void enumerated_ref_stop_async (struct enumerated_ref *);void enumerated_ref_stop (struct enumerated_ref *,const char *const []);void enumerated_ref_start (struct enumerated_ref *);
fs/bcachefs/fs-io.c +4 -2
@@ -49,7 +49,8 @@ static void nocow_flush_endio(struct bio *_bio) struct nocow_flush *bio = container_of(_bio, struct nocow_flush, bio);closure_put (bio->cl); -percpu_ref_put (&bio->ca->io_ref[WRITE ]); +enumerated_ref_put (&bio->ca->io_ref[WRITE ], + BCH_DEV_WRITE_REF_nocow_flush);bio_put (&bio->bio); }
@@ -72,7 +73,8 @@ void bch2_inode_flush_nocow_writes_async(struct bch_fs *c,for_each_set_bit (dev, devs.d,BCH_SB_MEMBERS_MAX ) {rcu_read_lock (); ca =rcu_dereference (c->devs[dev]); -if (ca && !percpu_ref_tryget (&ca->io_ref[WRITE ])) +if (ca && !enumerated_ref_tryget (&ca->io_ref[WRITE ], + BCH_DEV_WRITE_REF_nocow_flush)) ca =NULL ;rcu_read_unlock ();
fs/bcachefs/io_read.c +6 -4
@@ -411,7 +411,7 @@ static inline struct bch_read_bio *bch2_rbio_free(struct bch_read_bio *rbio)if (rbio->have_ioref) {struct bch_dev *ca =bch2_dev_have_ref (rbio->c, rbio->pick.ptr.dev); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_io_read); }if (rbio->split) {
@@ -1101,7 +1101,8 @@ int __bch2_read_extent(struct btree_trans *trans, struct bch_read_bio *orig, goto err; } -struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ ); +struct bch_dev *ca =bch2_dev_get_ioref (c, pick.ptr.dev,READ , + BCH_DEV_READ_REF_io_read);/* * Stale dirty pointers are treated as IO errors, but @failed isn't
@@ -1115,7 +1116,7 @@ int __bch2_read_extent(struct btree_trans *trans, struct bch_read_bio *orig, unlikely(dev_ptr_stale(ca, &pick.ptr))) {read_from_stale_dirty_pointer (trans, ca, k, pick.ptr);bch2_mark_io_failure (failed, &pick, false); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_io_read); goto retry_pick; }
@@ -1148,7 +1149,8 @@ int __bch2_read_extent(struct btree_trans *trans, struct bch_read_bio *orig,*/ if (pick.crc.compressed_size > u->op.wbio.bio.bi_iter.bi_size) {if (ca) -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], + BCH_DEV_READ_REF_io_read); rbio->ret = -BCH_ERR_data_read_buffer_too_small; goto out_read_done; }
fs/bcachefs/io_write.c +11 -4
@@ -442,6 +442,10 @@ void bch2_submit_wbio_replicas(struct bch_write_bio *wbio, struct bch_fs *c, {struct bkey_ptrs_c ptrs =bch2_bkey_ptrs_c (bkey_i_to_s_c (k));struct bch_write_bio *n; +unsigned ref_rw = type == BCH_DATA_btree ?READ :WRITE ; +unsigned ref_idx = type == BCH_DATA_btree + ? BCH_DEV_READ_REF_btree_node_write + : BCH_DEV_WRITE_REF_io_write;BUG_ON (c->opts.nochanges);
@@ -453,7 +457,7 @@ void bch2_submit_wbio_replicas(struct bch_write_bio *wbio, struct bch_fs *c,*/ struct bch_dev *ca = nocow ?bch2_dev_have_ref (c, ptr->dev) - :bch2_dev_get_ioref (c, ptr->dev, type == BCH_DATA_btree ?READ :WRITE ); + :bch2_dev_get_ioref (c, ptr->dev, ref_rw, ref_idx);if (to_entry (ptr +1 ) < ptrs.end) { n =to_wbio (bio_alloc_clone (NULL , &wbio->bio,GFP_NOFS , &c->replica_set));
@@ -728,7 +732,8 @@ static void bch2_write_endio(struct bio *bio) }if (wbio->have_ioref) -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], + BCH_DEV_WRITE_REF_io_write);if (wbio->bounce)bch2_bio_free_pages_pool (c, bio);
@@ -1325,7 +1330,8 @@ static void bch2_nocow_write(struct bch_write_op *op)/* Get iorefs before dropping btree locks: */ struct bkey_ptrs_c ptrs =bch2_bkey_ptrs_c (k);bkey_for_each_ptr (ptrs, ptr) { -struct bch_dev *ca =bch2_dev_get_ioref (c, ptr->dev,WRITE ); +struct bch_dev *ca =bch2_dev_get_ioref (c, ptr->dev,WRITE , + BCH_DEV_WRITE_REF_io_write);if (unlikely (!ca)) goto err_get_ioref;
@@ -1427,7 +1433,8 @@ static void bch2_nocow_write(struct bch_write_op *op)return ; err_get_ioref:darray_for_each (buckets, i) -percpu_ref_put (&bch2_dev_have_ref (c, i->b.inode)->io_ref[WRITE ]); +enumerated_ref_put (&bch2_dev_have_ref (c, i->b.inode)->io_ref[WRITE ], + BCH_DEV_WRITE_REF_io_write);/* Fall back to COW path: */ goto out;
fs/bcachefs/journal.c +3 -2
@@ -1336,13 +1336,14 @@ int bch2_dev_journal_alloc(struct bch_dev *ca, bool new_fs)int bch2_fs_journal_alloc (struct bch_fs *c) { -for_each_online_member (c, ca) { +for_each_online_member (c, ca, BCH_DEV_READ_REF_fs_journal_alloc) {if (ca->journal.nr)continue ;int ret =bch2_dev_journal_alloc (ca, true);if (ret) { -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], + BCH_DEV_READ_REF_fs_journal_alloc);return ret; } }
fs/bcachefs/journal_io.c +9 -6
@@ -1218,7 +1218,7 @@ static CLOSURE_CALLBACK(bch2_journal_read_device) out:bch_verbose (c,"journal read done on device %s, ret %i" , ca->name, ret);kvfree (buf.data); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_journal_read);closure_return (cl);return ; err:
@@ -1253,7 +1253,8 @@ int bch2_journal_read(struct bch_fs *c,if ((ca->mi.state == BCH_MEMBER_STATE_rw || ca->mi.state == BCH_MEMBER_STATE_ro) && -percpu_ref_tryget (&ca->io_ref[READ ])) +enumerated_ref_tryget (&ca->io_ref[READ ], + BCH_DEV_READ_REF_journal_read)) closure_call(&ca->journal.read, bch2_journal_read_device, system_unbound_wq,
@@ -1768,7 +1769,7 @@ static void journal_write_endio(struct bio *bio) }closure_put (&w->io); -percpu_ref_put (&ca->io_ref[WRITE ]); +enumerated_ref_put (&ca->io_ref[WRITE ], BCH_DEV_WRITE_REF_journal_write); }static CLOSURE_CALLBACK (journal_write_submit)
@@ -1779,7 +1780,8 @@ static CLOSURE_CALLBACK(journal_write_submit)unsigned sectors =vstruct_sectors (w->data, c->block_bits);extent_for_each_ptr (bkey_i_to_s_extent (&w->key), ptr) { -struct bch_dev *ca =bch2_dev_get_ioref (c, ptr->dev,WRITE ); +struct bch_dev *ca =bch2_dev_get_ioref (c, ptr->dev,WRITE , + BCH_DEV_WRITE_REF_journal_write);if (!ca) {/* XXX: fix this */ bch_err (c,"missing device for journal write\n" );
@@ -1842,8 +1844,9 @@ static CLOSURE_CALLBACK(journal_write_preflush) }if (w->separate_flush) { -for_each_rw_member (c, ca) { -percpu_ref_get (&ca->io_ref[WRITE ]); +for_each_rw_member (c, ca, BCH_DEV_WRITE_REF_journal_write) { +enumerated_ref_get (&ca->io_ref[WRITE ], + BCH_DEV_WRITE_REF_journal_write);struct journal_device *ja = &ca->journal;struct bio *bio = &ja->bio[w->idx]->bio;
fs/bcachefs/journal_reclaim.c +1 -1
@@ -285,7 +285,7 @@ void bch2_journal_do_discards(struct journal *j)mutex_lock (&j->discard_lock); -for_each_rw_member (c, ca) { +for_each_rw_member (c, ca, BCH_DEV_WRITE_REF_journal_do_discards) {struct journal_device *ja = &ca->journal;while (should_discard_bucket (j, ja)) {
fs/bcachefs/sb-members.h +17 -15
@@ -4,6 +4,7 @@#include "darray.h" #include "bkey_types.h" +#include "enumerated_ref.h" extern char *const bch2_member_error_strs[];
@@ -20,7 +21,7 @@ struct bch_member bch2_sb_member_get(struct bch_sb *sb, int i);static inline bool bch2_dev_is_online (struct bch_dev *ca) { -return !percpu_ref_is_zero (&ca->io_ref[READ ]); +return !enumerated_ref_is_zero (&ca->io_ref[READ ]); }static inline struct bch_dev *bch2_dev_rcu (struct bch_fs *,unsigned );
@@ -163,33 +164,33 @@ static inline struct bch_dev *bch2_get_next_dev(struct bch_fs *c, struct bch_devstatic inline struct bch_dev *bch2_get_next_online_dev (struct bch_fs *c,struct bch_dev *ca,unsigned state_mask, -int rw) +int rw,unsigned ref_idx) {rcu_read_lock ();if (ca) -percpu_ref_put (&ca->io_ref[rw]); +enumerated_ref_put (&ca->io_ref[rw], ref_idx);while ((ca =__bch2_next_dev (c, ca,NULL )) && (!((1 << ca->mi.state) & state_mask) || - !percpu_ref_tryget (&ca->io_ref[rw]))) + !enumerated_ref_tryget (&ca->io_ref[rw], ref_idx))) ;rcu_read_unlock ();return ca; } -#define __for_each_online_member (_c, _ca, state_mask, rw) \ +#define __for_each_online_member (_c, _ca, state_mask, rw, ref_idx) \ for (struct bch_dev *_ca = NULL; \ - (_ca = bch2_get_next_online_dev(_c, _ca, state_mask, rw));) + (_ca = bch2_get_next_online_dev(_c, _ca, state_mask, rw, ref_idx));) -#define for_each_online_member (c, ca) \ - __for_each_online_member(c, ca, ~0, READ) +#define for_each_online_member (c, ca, ref_idx) \ + __for_each_online_member(c, ca, ~0, READ, ref_idx) -#define for_each_rw_member (c, ca) \ - __for_each_online_member(c, ca, BIT(BCH_MEMBER_STATE_rw), WRITE) +#define for_each_rw_member (c, ca, ref_idx) \ + __for_each_online_member(c, ca, BIT(BCH_MEMBER_STATE_rw), WRITE, ref_idx) -#define for_each_readable_member (c, ca) \ - __for_each_online_member(c, ca, BIT( BCH_MEMBER_STATE_rw)|BIT(BCH_MEMBER_STATE_ro), READ) +#define for_each_readable_member (c, ca, ref_idx) \ + __for_each_online_member(c, ca, BIT( BCH_MEMBER_STATE_rw)|BIT(BCH_MEMBER_STATE_ro), READ, ref_idx)static inline bool bch2_dev_exists (const struct bch_fs *c,unsigned dev) {
@@ -290,13 +291,14 @@ static inline struct bch_dev *bch2_dev_iterate(struct bch_fs *c, struct bch_devreturn bch2_dev_tryget (c, dev_idx); } -static inline struct bch_dev *bch2_dev_get_ioref (struct bch_fs *c,unsigned dev,int rw) +static inline struct bch_dev *bch2_dev_get_ioref (struct bch_fs *c,unsigned dev, +int rw,unsigned ref_idx) {might_sleep ();rcu_read_lock ();struct bch_dev *ca =bch2_dev_rcu (c, dev); -if (ca && !percpu_ref_tryget (&ca->io_ref[rw])) +if (ca && !enumerated_ref_tryget (&ca->io_ref[rw], ref_idx)) ca =NULL ;rcu_read_unlock ();
@@ -306,7 +308,7 @@ static inline struct bch_dev *bch2_dev_get_ioref(struct bch_fs *c, unsigned dev,return ca;if (ca) -percpu_ref_put (&ca->io_ref[rw]); +enumerated_ref_put (&ca->io_ref[rw], ref_idx);return NULL ; }
fs/bcachefs/super-io.c +9 -9
@@ -260,11 +260,11 @@ struct bch_sb_field *bch2_sb_field_resize_id(struct bch_sb_handle *sb,/* XXX: we're not checking that offline device have enough space */ -for_each_online_member (c, ca) { +for_each_online_member (c, ca, BCH_DEV_READ_REF_sb_field_resize) {struct bch_sb_handle *dev_sb = &ca->disk_sb;if (bch2_sb_realloc (dev_sb,le32_to_cpu (dev_sb->sb->u64s) + d)) { -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_sb_field_resize);return NULL ; } }
@@ -967,7 +967,7 @@ static void write_super_endio(struct bio *bio) }closure_put (&ca->fs->sb_write); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_write_super); }static void read_back_super (struct bch_fs *c,struct bch_dev *ca)
@@ -985,7 +985,7 @@ static void read_back_super(struct bch_fs *c, struct bch_dev *ca)this_cpu_add (ca->io_done->sectors[READ ][BCH_DATA_sb],bio_sectors (bio)); -percpu_ref_get (&ca->io_ref[READ ]); +enumerated_ref_get (&ca->io_ref[READ ], BCH_DEV_READ_REF_write_super);closure_bio_submit (bio, &c->sb_write); }
@@ -1011,7 +1011,7 @@ static void write_one_super(struct bch_fs *c, struct bch_dev *ca, unsigned idx)this_cpu_add (ca->io_done->sectors[WRITE ][BCH_DATA_sb],bio_sectors (bio)); -percpu_ref_get (&ca->io_ref[READ ]); +enumerated_ref_get (&ca->io_ref[READ ], BCH_DEV_READ_REF_write_super);closure_bio_submit (bio, &c->sb_write); }
@@ -1043,13 +1043,13 @@ int bch2_write_super(struct bch_fs *c)* For now, we expect to be able to call write_super() when we're not * yet RW: */ -for_each_online_member (c, ca) { +for_each_online_member (c, ca, BCH_DEV_READ_REF_write_super) { ret =darray_push (&online_devices, ca);if (bch2_fs_fatal_err_on (ret, c,"%s: error allocating online devices" , __func__)) { -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_write_super); goto out; } -percpu_ref_get (&ca->io_ref[READ ]); +enumerated_ref_get (&ca->io_ref[READ ], BCH_DEV_READ_REF_write_super); }/* Make sure we're using the new magic numbers: */
@@ -1216,7 +1216,7 @@ int bch2_write_super(struct bch_fs *c)/* Make new options visible after they're persistent: */ bch2_sb_update (c);darray_for_each (online_devices, ca) -percpu_ref_put (&(*ca)->io_ref[READ ]); +enumerated_ref_put (&(*ca)->io_ref[READ ], BCH_DEV_READ_REF_write_super);darray_exit (&online_devices);printbuf_exit (&err);return ret;
fs/bcachefs/super.c +48 -45
212 lines not on this page · raw message
fs/bcachefs/super.h +3 -0
@@ -9,6 +9,9 @@#include <linux/math64.h> extern const char *const bch2_fs_flag_strs[]; +extern const char *const bch2_write_refs[]; +extern const char *const bch2_dev_read_refs[]; +extern const char *const bch2_dev_write_refs[];struct bch_fs *bch2_dev_to_fs (dev_t );struct bch_fs *bch2_uuid_to_fs (__uuid_t );
fs/bcachefs/sysfs.c +13 -9
@@ -178,14 +178,9 @@ read_attribute(btree_reserve_cache);read_attribute (open_buckets);read_attribute (open_buckets_partial);read_attribute (nocow_lock_table); -read_attribute (write_refs); -static const char *const bch2_write_refs[] = { -#define x(n) #n, -BCH_WRITE_REFS () -#undef x - NULL -}; +read_attribute (read_refs); +read_attribute (write_refs);read_attribute (internal_uuid);read_attribute (disk_groups);
@@ -314,7 +309,7 @@ static int bch2_read_fua_test(struct printbuf *out, struct bch_dev *ca)bch2_time_stats_init_no_pcpu (&stats_fua);bch2_time_stats_init_no_pcpu (&stats_random); -if (!bch2_dev_get_ioref (c, ca->dev_idx,READ )) { +if (!bch2_dev_get_ioref (c, ca->dev_idx,READ , BCH_DEV_READ_REF_read_fua_test)) {prt_str (out,"offline\n" );return 0 ; }
@@ -405,7 +400,7 @@ static int bch2_read_fua_test(struct printbuf *out, struct bch_dev *ca) err:kfree (buf);kfree (bio); -percpu_ref_put (&ca->io_ref[READ ]); +enumerated_ref_put (&ca->io_ref[READ ], BCH_DEV_READ_REF_read_fua_test);bch_err_fn (c, ret);return ret; }
@@ -905,6 +900,12 @@ SHOW(bch2_dev)if (opt_id >=0 )return sysfs_opt_show (c, ca, opt_id, out); +if (attr == &sysfs_read_refs) +enumerated_ref_to_text (out, &ca->io_ref[READ ], bch2_dev_read_refs); + +if (attr == &sysfs_write_refs) +enumerated_ref_to_text (out, &ca->io_ref[WRITE ], bch2_dev_write_refs); +return 0 ; }
@@ -962,6 +963,9 @@ struct attribute *bch2_dev_files[] = {/* debug: */ &sysfs_alloc_debug, &sysfs_open_buckets, + + &sysfs_read_refs, + &sysfs_write_refs, NULL };
Signature · 1 line
2.49.0
Quoted 4 lines · Kent Overstreet
Not sure we have a list for library code, but this might be of interest to anyone who's had to debug refcount issues on refs with lots of users (filesystem people), and I know the hardening folks deal with refcounts a lot.
Why not use refcount_t instead of atomic_t?
Signature · 1 line
Kees Cook
Quoted 7 lines · Kees Cook
Quoted 4 lines · Kent Overstreet
Not sure we have a list for library code, but this might be of interest to anyone who's had to debug refcount issues on refs with lots of users (filesystem people), and I know the hardening folks deal with refcounts a lot.
Why not use
refcount_tinstead ofatomic_t?
It's rather pointless here since percpu refcounts don't (and can't) support saturation, and atomic_long_t should always suffice - you'd have to be doing something particularly bizarre for it not to, since refcounts generally count things in memory.
Out of curiousity, has overflow of an atomic_long_t refcount ever been observed?
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Kees Cook
Quoted 4 lines · Kent Overstreet
Not sure we have a list for library code, but this might be of interest to anyone who's had to debug refcount issues on refs with lots of users (filesystem people), and I know the hardening folks deal with refcounts a lot.
Quoted 6 lines
Quoted 1 line
Why not use
refcount_tinstead ofatomic_t?It's rather pointless here since percpu refcounts don't (and can't) support saturation, and
atomic_long_tshould always suffice - you'd have to be doing something particularly bizarre for it not to, since refcounts generally count things in memory.
Ah yes, my eyes skipped over the "long" part when I was reading the patches. There's currently no sane reason to use refcount_t when already using atomic_long_t. Sorry for the noise!
Quoted 2 lines
Out of curiousity, has overflow of an
atomic_long_trefcount ever been observed?
Not to my knowledge. :)
Signature · 1 line
Kees Cook
Quoted 8 lines · Kees Cook
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Kees Cook
Quoted 4 lines · Kent Overstreet
Not sure we have a list for library code, but this might be of interest to anyone who's had to debug refcount issues on refs with lots of users (filesystem people), and I know the hardening folks deal with refcounts a lot.
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Why not use
refcount_tinstead ofatomic_t?Out of curiousity, has overflow of an
atomic_long_trefcount ever been observed?Not to my knowledge. :)
Equivalent systems have observed it, but only in the presence of compiler optimizations that deduce they could increment the refcount multiple times.
NEVER_INLINE void naughty_ref_increment(ref* ref) {
long i;
for (i = 0; i < LONG_MAX/2; i++) {
ref_get(ref);
}
}Running the above code 3 times will saturate the refcount, if it ever terminates in our lifetimes (due to being optimized into an atomic_fetch_add(LONG_MAX/2)).
So: don't write the above code!