Skip to contentSkip to conversation

bcache-next 20230615

PATCH Applied Coly Li 2 participants 8 messages Jun 15, 2023

Subsystems (4)
lore
Share & tools
lore

Coly Li

author
#0 lore raw

cover bcache-next 20230615

Hi Jens,

I start to follow Song Liu's -next style to submit bcache patches to you. This series are minor fixes I tested for a while, and generated based on top of the for-6.5/block branch from linux-block tree.

The patch from Mingzhe Zou fixes a race in bcache initializaiton time, rested patches from Andrea, Thomas, Zheng and Ye are code cleanup and good to have them in.

Please consider taking them, and thank you in advance.

Coly Li
---

Andrea Tomassetti (1):
bcache: Remove dead references to cache_readaheads

Mingzhe Zou (1):
bcache: fixup btree_cache_wait list damage

Thomas Weißschuh (1):
bcache: make kobj_type structures constant

Zheng Wang (2):
bcache: Remove some unnecessary NULL point check for the return value
of __bch_btree_node_alloc-related pointer
bcache: Fix __bch_btree_node_alloc to make the failure behavior
consistent

ye xingchen (1):
bcache: Convert to use sysfs_emit()/sysfs_emit_at() APIs

 Documentation/admin-guide/bcache.rst |  3 ---
 drivers/md/bcache/bcache.h           | 10 ++++-----
 drivers/md/bcache/btree.c            | 25 +++++++++++++++-------
 drivers/md/bcache/btree.h            |  1 +
 drivers/md/bcache/stats.h            |  1 -
 drivers/md/bcache/super.c            |  4 ++--
 drivers/md/bcache/sysfs.c            | 31 ++++++++++++++--------------
 drivers/md/bcache/sysfs.h            |  2 +-
 drivers/md/bcache/writeback.c        | 10 +++++++++
 9 files changed, 52 insertions(+), 35 deletions(-)
Signature · 1 line
2.35.3
author
#1 lore raw

PATCH 1/6 bcache: Convert to use sysfs_emit()/sysfs_emit_at() APIs

From: ye xingchen <[email protected]>

Follow the advice of the Documentation/filesystems/sysfs.rst and show() should only use sysfs_emit() or sysfs_emit_at() when formatting the value to be returned to user space.

---
 drivers/md/bcache/sysfs.c | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)
drivers/md/bcache/sysfs.c +15 -16
index c6f677059214..0e2c1880f60b 100644
@@ -1111,26 +1111,25 @@ SHOW(__bch_cache)
 
 		vfree(p);
 
-		ret = scnprintf(buf, PAGE_SIZE,
-				"Unused:		%zu%%\n"
-				"Clean:		%zu%%\n"
-				"Dirty:		%zu%%\n"
-				"Metadata:	%zu%%\n"
-				"Average:	%llu\n"
-				"Sectors per Q:	%zu\n"
-				"Quantiles:	[",
-				unused * 100 / (size_t) ca->sb.nbuckets,
-				available * 100 / (size_t) ca->sb.nbuckets,
-				dirty * 100 / (size_t) ca->sb.nbuckets,
-				meta * 100 / (size_t) ca->sb.nbuckets, sum,
-				n * ca->sb.bucket_size / (ARRAY_SIZE(q) + 1));
+		ret = sysfs_emit(buf,
+				 "Unused:		%zu%%\n"
+				 "Clean:		%zu%%\n"
+				 "Dirty:		%zu%%\n"
+				 "Metadata:	%zu%%\n"
+				 "Average:	%llu\n"
+				 "Sectors per Q:	%zu\n"
+				 "Quantiles:	[",
+				 unused * 100 / (size_t) ca->sb.nbuckets,
+				 available * 100 / (size_t) ca->sb.nbuckets,
+				 dirty * 100 / (size_t) ca->sb.nbuckets,
+				 meta * 100 / (size_t) ca->sb.nbuckets, sum,
+				 n * ca->sb.bucket_size / (ARRAY_SIZE(q) + 1));
 
 		for (i = 0; i < ARRAY_SIZE(q); i++)
-			ret += scnprintf(buf + ret, PAGE_SIZE - ret,
-					 "%u ", q[i]);
+			ret += sysfs_emit_at(buf, ret, "%u ", q[i]);
 		ret--;
 
-		ret += scnprintf(buf + ret, PAGE_SIZE - ret, "]\n");
+		ret += sysfs_emit_at(buf, ret, "]\n");
 
 		return ret;
 	}
Signature · 1 line
2.35.3
author
#2 lore raw

PATCH 2/6 bcache: make kobj_type structures constant

In reply to Coly Li

From: Thomas Weißschuh <[email protected]>

Since commit ee6d3dd4ed48 ("driver core: make kobj_type constant.") the driver core allows the usage of const struct kobj_type.

Take advantage of this to constify the structure definitions to prevent modification at runtime.

---
 drivers/md/bcache/bcache.h | 10 +++++-----
 drivers/md/bcache/sysfs.h  |  2 +-
 2 files changed, 6 insertions(+), 6 deletions(-)
drivers/md/bcache/bcache.h +5 -5
index aebb7ef10e63..a522f4f1f992 100644
@@ -1004,11 +1004,11 @@ extern struct workqueue_struct *bch_flush_wq;
 extern struct mutex bch_register_lock;
 extern struct list_head bch_cache_sets;
 
-extern struct kobj_type bch_cached_dev_ktype;
-extern struct kobj_type bch_flash_dev_ktype;
-extern struct kobj_type bch_cache_set_ktype;
-extern struct kobj_type bch_cache_set_internal_ktype;
-extern struct kobj_type bch_cache_ktype;
+extern const struct kobj_type bch_cached_dev_ktype;
+extern const struct kobj_type bch_flash_dev_ktype;
+extern const struct kobj_type bch_cache_set_ktype;
+extern const struct kobj_type bch_cache_set_internal_ktype;
+extern const struct kobj_type bch_cache_ktype;
 
 void bch_cached_dev_release(struct kobject *kobj);
 void bch_flash_dev_release(struct kobject *kobj);
drivers/md/bcache/sysfs.h +1 -1
index a2ff6447b699..65b8bd975ab1 100644
@@ -3,7 +3,7 @@
 #define _BCACHE_SYSFS_H_
 
 #define KTYPE(type)							\
-struct kobj_type type ## _ktype = {					\
+const struct kobj_type type ## _ktype = {					\
 	.release	= type ## _release,				\
 	.sysfs_ops	= &((const struct sysfs_ops) {			\
 		.show	= type ## _show,				\
Signature · 1 line
2.35.3
author
#3 lore raw

PATCH 3/6 bcache: Remove dead references to cache_readaheads

In reply to Coly Li

From: Andrea Tomassetti <[email protected]>

The cache_readaheads stat counter is not used anymore and should be removed.

---
 Documentation/admin-guide/bcache.rst | 3 ---
 drivers/md/bcache/stats.h            | 1 -
 2 files changed, 4 deletions(-)
Documentation/admin-guide/bcache.rst +0 -3
index bb5032a99234..6fdb495ac466 100644
@@ -508,9 +508,6 @@ cache_miss_collisions
   cache miss, but raced with a write and data was already present (usually 0
   since the synchronization for cache misses was rewritten)
 
-cache_readaheads
-  Count of times readahead occurred.
-
 Sysfs - cache set
 ~~~~~~~~~~~~~~~~~
 
drivers/md/bcache/stats.h +0 -1
index bd3afc856d53..21b445f8af15 100644
@@ -18,7 +18,6 @@ struct cache_stats {
 	unsigned long cache_misses;
 	unsigned long cache_bypass_hits;
 	unsigned long cache_bypass_misses;
-	unsigned long cache_readaheads;
 	unsigned long cache_miss_collisions;
 	unsigned long sectors_bypassed;
 
Signature · 1 line
2.35.3
author
#4 lore raw

PATCH 4/6 bcache: Remove some unnecessary NULL point check for the return value of __bch_btree_node_alloc-related pointer

In reply to Coly Li

From: Zheng Wang <[email protected]>

Due to the previously fix of __bch_btree_node_alloc, the return value will never be a NULL pointer. So IS_ERR is enough to handle the failure
situation. Fix it by replacing IS_ERR_OR_NULL check to IS_ERR check.

Fixes cafe56359144 ("bcache: A block layer cache")
---
 drivers/md/bcache/btree.c | 10 +++++-----
 drivers/md/bcache/super.c |  4 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)
drivers/md/bcache/btree.c +5 -5
index 147c493a989a..7c21e54468bf 100644
@@ -1138,7 +1138,7 @@ static struct btree *btree_node_alloc_replacement(struct btree *b,
 {
 	struct btree *n = bch_btree_node_alloc(b->c, op, b->level, b->parent);
 
-	if (!IS_ERR_OR_NULL(n)) {
+	if (!IS_ERR(n)) {
 		mutex_lock(&n->write_lock);
 		bch_btree_sort_into(&b->keys, &n->keys, &b->c->sort);
 		bkey_copy_key(&n->key, &b->key);
@@ -1340,7 +1340,7 @@ static int btree_gc_coalesce(struct btree *b, struct btree_op *op,
 	memset(new_nodes, 0, sizeof(new_nodes));
 	closure_init_stack(&cl);
 
-	while (nodes < GC_MERGE_NODES && !IS_ERR_OR_NULL(r[nodes].b))
+	while (nodes < GC_MERGE_NODES && !IS_ERR(r[nodes].b))
 		keys += r[nodes++].keys;
 
 	blocks = btree_default_blocks(b->c) * 2 / 3;
@@ -1352,7 +1352,7 @@ static int btree_gc_coalesce(struct btree *b, struct btree_op *op,
 
 	for (i = 0; i < nodes; i++) {
 		new_nodes[i] = btree_node_alloc_replacement(r[i].b, NULL);
-		if (IS_ERR_OR_NULL(new_nodes[i]))
+		if (IS_ERR(new_nodes[i]))
 			goto out_nocoalesce;
 	}
 
@@ -1487,7 +1487,7 @@ static int btree_gc_coalesce(struct btree *b, struct btree_op *op,
 	bch_keylist_free(&keylist);
 
 	for (i = 0; i < nodes; i++)
-		if (!IS_ERR_OR_NULL(new_nodes[i])) {
+		if (!IS_ERR(new_nodes[i])) {
 			btree_node_free(new_nodes[i]);
 			rw_unlock(true, new_nodes[i]);
 		}
@@ -1669,7 +1669,7 @@ static int bch_btree_gc_root(struct btree *b, struct btree_op *op,
 	if (should_rewrite) {
 		n = btree_node_alloc_replacement(b, NULL);
 
-		if (!IS_ERR_OR_NULL(n)) {
+		if (!IS_ERR(n)) {
 			bch_btree_node_write_sync(n);
 
 			bch_btree_set_root(n);
drivers/md/bcache/super.c +2 -2
index 7e9d19fd21dd..077149c4050b 100644
@@ -1723,7 +1723,7 @@ static void cache_set_flush(struct closure *cl)
 	if (!IS_ERR_OR_NULL(c->gc_thread))
 		kthread_stop(c->gc_thread);
 
-	if (!IS_ERR_OR_NULL(c->root))
+	if (!IS_ERR(c->root))
 		list_add(&c->root->list, &c->btree_cache);
 
 	/*
@@ -2087,7 +2087,7 @@ static int run_cache_set(struct cache_set *c)
 
 		err = "cannot allocate new btree root";
 		c->root = __bch_btree_node_alloc(c, NULL, 0, true, NULL);
-		if (IS_ERR_OR_NULL(c->root))
+		if (IS_ERR(c->root))
 			goto err;
 
 		mutex_lock(&c->root->write_lock);
Signature · 1 line
2.35.3
author
#5 lore raw

PATCH 5/6 bcache: Fix __bch_btree_node_alloc to make the failure behavior consistent

In reply to Coly Li

From: Zheng Wang <[email protected]>

In some specific situation, the return value of __bch_btree_node_alloc may be NULL. This may lead to poential NULL pointer dereference in caller
function like a calling chaion :
btree_split->bch_btree_node_alloc->__bch_btree_node_alloc.

Fix it by initialize return value in __bch_btree_node_alloc before return.

Fixes cafe56359144 ("bcache: A block layer cache")
---
 drivers/md/bcache/btree.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/md/bcache/btree.c +3 -1
index 7c21e54468bf..0ddf91204782 100644
@@ -1090,10 +1090,12 @@ struct btree *__bch_btree_node_alloc(struct cache_set *c, struct btree_op *op,
 				     struct btree *parent)
 {
 	BKEY_PADDED(key) k;
-	struct btree *b = ERR_PTR(-EAGAIN);
+	struct btree *b;
 
 	mutex_lock(&c->bucket_lock);
 retry:
+	/* return ERR_PTR(-EAGAIN) when it fails */
+	b = ERR_PTR(-EAGAIN);
 	if (__bch_bucket_alloc_set(c, RESERVE_BTREE, &k.key, wait))
 		goto err;
 
Signature · 1 line
2.35.3
author
#6 lore raw

PATCH 6/6 bcache: fixup btree_cache_wait list damage

In reply to Coly Li

From: Mingzhe Zou <[email protected]>

We get a kernel crash about "list_add corruption. next->prev should be prev (ffff9c801bc01210), but was ffff9c77b688237c. (next=ffffae586d8afe68)."

crash> struct list_head 0xffff9c801bc01210
struct list_head {
next = 0xffffae586d8afe68,
prev = 0xffffae586d8afe68
}
crash> struct list_head 0xffff9c77b688237c
struct list_head {
next = 0x0,
prev = 0x0
}
crash> struct list_head 0xffffae586d8afe68
struct list_head struct: invalid kernel virtual address: ffffae586d8afe68 type: "gdb_readmem_callback" Cannot access memory at address 0xffffae586d8afe68

[230469.019492] Call Trace:
[230469.032041]  prepare_to_wait+0x8a/0xb0
[230469.044363]  ? bch_btree_keys_free+0x6c/0xc0 [escache]
[230469.056533]  mca_cannibalize_lock+0x72/0x90 [escache]
[230469.068788]  mca_alloc+0x2ae/0x450 [escache]
[230469.080790]  bch_btree_node_get+0x136/0x2d0 [escache]
[230469.092681]  bch_btree_check_thread+0x1e1/0x260 [escache]
[230469.104382]  ? finish_wait+0x80/0x80
[230469.115884]  ? bch_btree_check_recurse+0x1a0/0x1a0 [escache]
[230469.127259]  kthread+0x112/0x130
[230469.138448]  ? kthread_flush_work_fn+0x10/0x10
[230469.149477]  ret_from_fork+0x35/0x40

bch_btree_check_thread() and bch_dirty_init_thread() maybe call mca_cannibalize() to cannibalize other cached btree nodes. Only one thread can do it at a time, so the op of other threads will be added to the btree_cache_wait list.

We must call finish_wait() to remove op from btree_cache_wait before free it's memory address. Otherwise, the list will be damaged. Also should call bch_cannibalize_unlock() to release the btree_cache_alloc_lock and wake_up other waiters.

Fixes 8e7102273f59 ("bcache: make bch_btree_check() to be multithreaded")
Fixes b144e45fc576 ("bcache: make bch_sectors_dirty_init() to be multithreaded")
---
 drivers/md/bcache/btree.c     | 11 ++++++++++-
 drivers/md/bcache/btree.h     |  1 +
 drivers/md/bcache/writeback.c | 10 ++++++++++
 3 files changed, 21 insertions(+), 1 deletion(-)
drivers/md/bcache/btree.c +10 -1
index 0ddf91204782..68b9d7ca864e 100644
@@ -885,7 +885,7 @@ static struct btree *mca_cannibalize(struct cache_set *c, struct btree_op *op,
  * cannibalize_bucket() will take. This means every time we unlock the root of
  * the btree, we need to release this lock if we have it held.
  */
-static void bch_cannibalize_unlock(struct cache_set *c)
+void bch_cannibalize_unlock(struct cache_set *c)
 {
 	spin_lock(&c->btree_cannibalize_lock);
 	if (c->btree_cache_alloc_lock == current) {
@@ -1970,6 +1970,15 @@ static int bch_btree_check_thread(void *arg)
 			c->gc_stats.nodes++;
 			bch_btree_op_init(&op, 0);
 			ret = bcache_btree(check_recurse, p, c->root, &op);
+			/*
+			 * The op may be added to cache_set's btree_cache_wait
+			 * in mca_cannibalize(), must ensure it is removed from
+			 * the list and release btree_cache_alloc_lock before
+			 * free op memory.
+			 * Otherwise, the btree_cache_wait will be damaged.
+			 */
+			bch_cannibalize_unlock(c);
+			finish_wait(&c->btree_cache_wait, &(&op)->wait);
 			if (ret)
 				goto out;
 		}
drivers/md/bcache/btree.h +1 -0
index 1b5fdbc0d83e..a2920bbfcad5 100644
@@ -282,6 +282,7 @@ void bch_initial_gc_finish(struct cache_set *c);
 void bch_moving_gc(struct cache_set *c);
 int bch_btree_check(struct cache_set *c);
 void bch_initial_mark_key(struct cache_set *c, int level, struct bkey *k);
+void bch_cannibalize_unlock(struct cache_set *c);
 
 static inline void wake_up_gc(struct cache_set *c)
 {
drivers/md/bcache/writeback.c +10 -0
index d4a5fc0650bb..24c049067f61 100644
@@ -890,6 +890,16 @@ static int bch_root_node_dirty_init(struct cache_set *c,
 	if (ret < 0)
 		pr_warn("sectors dirty init failed, ret=%d!\n", ret);
 
+	/*
+	 * The op may be added to cache_set's btree_cache_wait
+	 * in mca_cannibalize(), must ensure it is removed from
+	 * the list and release btree_cache_alloc_lock before
+	 * free op memory.
+	 * Otherwise, the btree_cache_wait will be damaged.
+	 */
+	bch_cannibalize_unlock(c);
+	finish_wait(&c->btree_cache_wait, &(&op.op)->wait);
+
 	return ret;
 }
 
Signature · 1 line
2.35.3
maintainer Applied
#7 lore raw
In reply to Coly Li
Quoted 9 lines · Coly Li

I start to follow Song Liu's -next style to submit bcache patches to you. This series are minor fixes I tested for a while, and generated based on top of the for-6.5/block branch from linux-block tree.

The patch from Mingzhe Zou fixes a race in bcache initializaiton time, rested patches from Andrea, Thomas, Zheng and Ye are code cleanup and good to have them in.

[...]

Applied, thanks!

[1/6] bcache: Convert to use sysfs_emit()/sysfs_emit_at() APIs

      commit: a301b2deb66cd93bae0f676702356273ebf8abb6
[2/6] bcache: make kobj_type structures constant
      commit: b98dd0b0a596fdeaca68396ce8f782883ed253a9
[3/6] bcache: Remove dead references to cache_readaheads
      commit: ccb8c3bd6d93e7986b702d1f66d5d56d08abc59f
[4/6] bcache: Remove some unnecessary NULL point check for the return value of __bch_btree_node_alloc-related pointer
      commit: 028ddcac477b691dd9205c92f991cc15259d033e
[5/6] bcache: Fix __bch_btree_node_alloc to make the failure behavior consistent
      commit: 80fca8a10b604afad6c14213fdfd816c4eda3ee4
[6/6] bcache: fixup btree_cache_wait list damage
      commit: f0854489fc07d2456f7cc71a63f4faf9c716ffbe

Best regards,

Signature · 1 line
Jens Axboe