Skip to contentSkip to conversation

ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown.patch removed from -mm tree

merged mm-nonmm-stable Andrew Morton 1 participant 1 message Aug 4

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: ocfs2: synchronize heartbeat callbacks with o2net teardown
has been removed from the -mm tree. Its filename was
ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown.patch

This patch was dropped because it was merged into the mm-nonmm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: Cen Zhang <[email protected]>
Subject: ocfs2: synchronize heartbeat callbacks with o2net teardown
Date: Wed, 24 Jun 2026 17:53:09 +0800

Patch series "ocfs2: harden heartbeat teardown races".

This series fixes two OCFS2 heartbeat/o2net teardown races found by KASAN.

This patch (of 2):

Heartbeat callbacks stay registered while configfs local-node teardown enters o2net_stop_listening(). A node-down event can still run through o2net_disconnect_node() and o2net_set_nn_state() while teardown is destroying o2net_wq, so the later queue/flush operations can hit a dead workqueue. KASAN has caught this as a slab-use-after-free in __queue_work() with the call chain:

KASAN slab-use-after-free in __queue_work+0x56/0xa90

Read of size 4
Call trace:
  dump_stack_lvl+0x66/0xa0
  print_report+0xce/0x630
  __queue_work+0x56/0xa90
  srso_alias_return_thunk+0x5/0xfbef5
  __virt_addr_valid+0x19f/0x330
  kasan_report+0xe0/0x110
  __queue_delayed_work+0x58/0x1e0
  queue_delayed_work_on+0xb4/0xc0
  o2net_set_nn_state+0x467/0x840
  o2net_disconnect_node+0x7b/0xe0
  o2net_hb_node_down_cb+0x54/0x60
  o2hb_run_event_list+0x236/0x2d0
  o2hb_check_slot+0xad4/0xbc0
  lock_release+0xc8/0x290
  o2hb_check_slot+0x9ea/0xbc0
  trace_hardirqs_on+0x18/0x130
  o2hb_do_disk_heartbeat+0x646/0xb30 (fs/ocfs2/cluster/heartbeat.c:1079)
  __lock_acquire+0x466/0x2260
  lockdep_hardirqs_on_prepare+0xea/0x1a0
  ktime_get_with_offset+0xe9/0x230
  o2hb_thread+0x14e/0x770
  kthread+0x1ad/0x1f0
  ret_from_fork+0x3c9/0x540
  __switch_to+0x2e9/0x730
  ret_from_fork_asm+0x1a/0x30
Allocated by task stack:
  kasan_save_stack+0x33/0x60
  kasan_save_track+0x14/0x30
  __kasan_kmalloc+0xaa/0xb0
  __kmalloc_noprof+0x292/0x760
  __alloc_workqueue+0x736/0xc60
  alloc_workqueue_noprof+0xb1/0x110
  o2net_start_listening+0xe5/0x430
  o2nm_node_local_store+0x184/0x310
  configfs_write_iter+0x18a/0x210
  vfs_write+0x469/0x810
  ksys_write+0xd2/0x170
  do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
  entry_SYSCALL_64_after_hwframe+0x77/0x7f
Freed by task stack:
  kasan_save_stack+0x33/0x60
  kasan_save_track+0x14/0x30
  kasan_save_free_info+0x3b/0x60
  __kasan_slab_free+0x5f/0x80
  kfree+0x313/0x590
  rcu_core+0x4f4/0x1320
  handle_softirqs+0x156/0x660

queue_delayed_work_on
o2net_set_nn_state
o2net_disconnect_node
o2net_hb_node_down_cb
o2hb_run_event_list

Keep heartbeat callbacks registered so quorum state still tracks node state, but stop them from driving o2net reconnect/disconnect work once local teardown starts. Mark the transport offline before destroying o2net_wq, wait for any in-flight heartbeat callback to finish, and delay bring-up replay until the new local node is published through o2nm_this_node().

The replay also has to stay serialized with heartbeat callback delivery. Otherwise a live-node snapshot can be copied, a real hb_down callback can install -ENOTCONN for a peer, and the stale replay can call o2net_hb_node_up() for that same peer and queue reconnect work even though heartbeat is already down.

The buggy scenario involves two paths, with each column showing the order within that path:

local-node teardown: heartbeat node-down callback:
1. configfs local-off enters 1. o2hb_run_event_list() invokes
o2net_stop_listening(). o2net_hb_node_down_cb().
2. teardown heads for 2. the callback reaches
destroy_workqueue(o2net_wq). o2net_disconnect_node() and

                                        o2net_set_nn_state().

3. teardown destroys and NULLs 3. the callback flushes or queues
o2net_wq. work through o2net_wq.

Fixes 98211489d414 ("[PATCH] OCFS2: The Second Oracle Cluster Filesystem")
Assisted-by Codex:gpt-5.5
Cc Changwei Ge <[email protected]>
Cc Heming Zhao <[email protected]>
Cc Joel Becker <[email protected]>
Cc Jun Piao <[email protected]>
Cc Junxiao Bi <[email protected]>
Cc Mark Fasheh <[email protected]>

---

 fs/ocfs2/cluster/heartbeat.c   |   43 ++++++++++++----
 fs/ocfs2/cluster/heartbeat.h   |    5 +
 fs/ocfs2/cluster/nodemanager.c |    4 +
 fs/ocfs2/cluster/tcp.c         |   83 ++++++++++++++++++++++++-------
 fs/ocfs2/cluster/tcp.h         |    1 
 5 files changed, 109 insertions(+), 27 deletions(-)
fs/ocfs2/cluster/heartbeat.c renamed from fs/ocfs2/cluster/heartbeat.c~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +33 -10
@@ -1492,13 +1492,38 @@ void o2hb_init(void)
 	o2hb_debug_init();
 }
 
-/* if we're already in a callback then we're already serialized by the sem */
-static void o2hb_fill_node_map_from_callback(unsigned long *map,
-					     unsigned int bits)
+static void __o2hb_fill_node_map(unsigned long *map, unsigned int bits)
 {
 	bitmap_copy(map, o2hb_live_node_bitmap, bits);
 }
 
+void o2hb_callback_read_lock(void)
+{
+	down_read(&o2hb_callback_sem);
+}
+
+void o2hb_callback_read_unlock(void)
+{
+	up_read(&o2hb_callback_sem);
+}
+
+void o2hb_synchronize_callbacks(void)
+{
+	down_write(&o2hb_callback_sem);
+	up_write(&o2hb_callback_sem);
+}
+
+/*
+ * Callers must already hold o2hb_callback_sem for read or write so the copy
+ * stays serialized with callback delivery.
+ */
+void o2hb_fill_node_map_locked(unsigned long *map, unsigned int bits)
+{
+	spin_lock(&o2hb_live_lock);
+	__o2hb_fill_node_map(map, bits);
+	spin_unlock(&o2hb_live_lock);
+}
+
 /*
  * get a map of all nodes that are heartbeating in any regions
  */
@@ -1506,11 +1531,9 @@ void o2hb_fill_node_map(unsigned long *m
 {
 	/* callers want to serialize this map and callbacks so that they
 	 * can trust that they don't miss nodes coming to the party */
-	down_read(&o2hb_callback_sem);
-	spin_lock(&o2hb_live_lock);
-	o2hb_fill_node_map_from_callback(map, bits);
-	spin_unlock(&o2hb_live_lock);
-	up_read(&o2hb_callback_sem);
+	o2hb_callback_read_lock();
+	o2hb_fill_node_map_locked(map, bits);
+	o2hb_callback_read_unlock();
 }
 EXPORT_SYMBOL_GPL(o2hb_fill_node_map);
 
@@ -2609,7 +2632,7 @@ int o2hb_check_node_heartbeating_no_sem(
 	unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)];
 
 	spin_lock(&o2hb_live_lock);
-	o2hb_fill_node_map_from_callback(testing_map, O2NM_MAX_NODES);
+	__o2hb_fill_node_map(testing_map, O2NM_MAX_NODES);
 	spin_unlock(&o2hb_live_lock);
 	if (!test_bit(node_num, testing_map)) {
 		mlog(ML_HEARTBEAT,
@@ -2626,7 +2649,7 @@ int o2hb_check_node_heartbeating_from_ca
 {
 	unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)];
 
-	o2hb_fill_node_map_from_callback(testing_map, O2NM_MAX_NODES);
+	o2hb_fill_node_map_locked(testing_map, O2NM_MAX_NODES);
 	if (!test_bit(node_num, testing_map)) {
 		mlog(ML_HEARTBEAT,
 		     "node (%u) does not have heartbeating enabled.\n",
fs/ocfs2/cluster/heartbeat.h renamed from fs/ocfs2/cluster/heartbeat.h~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +5 -0
@@ -58,6 +58,11 @@ int o2hb_register_callback(const char *r
 			   struct o2hb_callback_func *hc);
 void o2hb_unregister_callback(const char *region_uuid,
 			      struct o2hb_callback_func *hc);
+void o2hb_callback_read_lock(void);
+void o2hb_callback_read_unlock(void);
+void o2hb_synchronize_callbacks(void);
+void o2hb_fill_node_map_locked(unsigned long *map,
+			       unsigned int bits);
 void o2hb_fill_node_map(unsigned long *map,
 			unsigned int bits);
 void o2hb_exit(void);
fs/ocfs2/cluster/nodemanager.c renamed from fs/ocfs2/cluster/nodemanager.c~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +4 -0
@@ -326,6 +326,7 @@ static ssize_t o2nm_node_local_store(str
 	struct o2nm_node *node = to_o2nm_node(item);
 	struct o2nm_cluster *cluster;
 	unsigned long tmp;
+	bool starting = false;
 	char *p = (char *)page;
 	ssize_t ret;
 
@@ -362,6 +363,7 @@ static ssize_t o2nm_node_local_store(str
 		ret = o2net_start_listening(node);
 		if (ret)
 			goto out;
+		starting = true;
 	}
 
 	if (!tmp && cluster->cl_has_local &&
@@ -375,6 +377,8 @@ static ssize_t o2nm_node_local_store(str
 	if (node->nd_local) {
 		cluster->cl_has_local = tmp;
 		cluster->cl_local_node = node->nd_num;
+		if (starting)
+			o2net_complete_start_listening(node);
 	}
 
 	ret = count;
fs/ocfs2/cluster/tcp.c renamed from fs/ocfs2/cluster/tcp.c~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +66 -17

137 lines not on this page · raw message

fs/ocfs2/cluster/tcp.h renamed from fs/ocfs2/cluster/tcp.h~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +1 -0
@@ -96,6 +96,7 @@ struct o2nm_node;
 int o2net_register_hb_callbacks(void);
 void o2net_unregister_hb_callbacks(void);
 int o2net_start_listening(struct o2nm_node *node);
+void o2net_complete_start_listening(struct o2nm_node *node);
 void o2net_stop_listening(struct o2nm_node *node);
 void o2net_disconnect_node(struct o2nm_node *node);
 int o2net_num_connected_peers(void);

_

Patches currently in -mm which might be from [email protected] are