Few bug fixes

Patch fixes a bug in the o2dlm protocol negotiation in that it is using the builtin version rather than the negotiated version during the domain join. This causes join errors when a node having kernel >= 2.6.37 joins a cluster with nodes having kernels < 2.6.37.

This only affects the o2cb cluster stack.

Signed-off-by Sunil Mushran <[email protected]>

Reported-by Jacek Stepniewski <[email protected]>

Double the live threshold for the first region in the global heartbeat mode. The default, 2, is bare minimum. Increasing it will affect all mounts in the local heartbeat mode. Instead we increase it only for the global heartbeat mode and that too only the first region. This is only to increase the margin of safety.

Addresses internal Oracle bug#10635585.

Signed-off-by Sunil Mushran <[email protected]>

---
 fs/ocfs2/cluster/heartbeat.c |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

fs/ocfs2/cluster/heartbeat.c +12 -1

index 2461eb3..ec50121 100644
@@ -1690,6 +1690,7 @@ static ssize_t o2hb_region_dev_write(struct o2hb_region *reg,
 	struct file *filp = NULL;
 	struct inode *inode = NULL;
 	ssize_t ret = -EINVAL;
+	int live_threshold;
 
 	if (reg->hr_bdev)
 		goto out;

@@ -1766,8 +1767,18 @@ static ssize_t o2hb_region_dev_write(struct o2hb_region *reg,
 	 * A node is considered live after it has beat LIVE_THRESHOLD
 	 * times.  We're not steady until we've given them a chance
 	 * _after_ our first read.
+	 * The default threshold is bare minimum so as to limit the delay
+	 * during mounts. For global heartbeat, the threshold doubled for the
+	 * first region.
 	 */
-	atomic_set(&reg->hr_steady_iterations, O2HB_LIVE_THRESHOLD + 1);
+	live_threshold = O2HB_LIVE_THRESHOLD;
+	if (o2hb_global_heartbeat_active()) {
+		spin_lock(&o2hb_live_lock);
+		if (o2hb_pop_count(&o2hb_region_bitmap, O2NM_MAX_REGIONS) == 1)
+			live_threshold <<= 1;
+		spin_unlock(&o2hb_live_lock);
+	}
+	atomic_set(&reg->hr_steady_iterations, live_threshold + 1);
 
 	hb_task = kthread_run(o2hb_thread, reg, "o2hb-%s",
 			      reg->hr_item.ci_name);

Signature · 7 lines

1.7.1


_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
http://oss.oracle.com/mailman/listinfo/ocfs2-devel

The patch itself looks fine. Can I ask you to add more reasoning for the change in the comment? Pretend that we're looking at that code in 2 years scratching our heads going "why did this get changed"

	--Mark

Quoted 53 lines · Sunil Mushran

Double the live threshold for the first region in the global heartbeat mode. The default, 2, is bare minimum. Increasing it will affect all mounts in the local heartbeat mode. Instead we increase it only for the global heartbeat mode and that too only the first region. This is only to increase the margin of safety.

Addresses internal Oracle bug#10635585.

Signed-off-by Sunil Mushran<[email protected]>

---
  fs/ocfs2/cluster/heartbeat.c |   13 ++++++++++++-
  1 files changed, 12 insertions(+), 1 deletions(-)

fs/ocfs2/cluster/heartbeat.c +12 -1

index 2461eb3..ec50121 100644
@@ -1690,6 +1690,7 @@ static ssize_t o2hb_region_dev_write(struct o2hb_region *reg,
  	struct file *filp = NULL;
  	struct inode *inode = NULL;
  	ssize_t ret = -EINVAL;
+	int live_threshold;
 
  	if (reg->hr_bdev)
  		goto out;

@@ -1766,8 +1767,18 @@ static ssize_t o2hb_region_dev_write(struct o2hb_region *reg,
  	 * A node is considered live after it has beat LIVE_THRESHOLD
  	 * times.  We're not steady until we've given them a chance
  	 * _after_ our first read.
+	 * The default threshold is bare minimum so as to limit the delay
+	 * during mounts. For global heartbeat, the threshold doubled for the
+	 * first region.
  	 */
-	atomic_set(&reg->hr_steady_iterations, O2HB_LIVE_THRESHOLD + 1);
+	live_threshold = O2HB_LIVE_THRESHOLD;
+	if (o2hb_global_heartbeat_active()) {
+		spin_lock(&o2hb_live_lock);
+		if (o2hb_pop_count(&o2hb_region_bitmap, O2NM_MAX_REGIONS) == 1)
+			live_threshold<<= 1;
+		spin_unlock(&o2hb_live_lock);
+	}
+	atomic_set(&reg->hr_steady_iterations, live_threshold + 1);
 
  	hb_task = kthread_run(o2hb_thread, reg, "o2hb-%s",
  			      reg->hr_item.ci_name);

Signature · 7 lines

1.7.1


_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
http://oss.oracle.com/mailman/listinfo/ocfs2-devel

--
Mark Fasheh

We have seen isolated cases (very few, I might add) of o2hb not detecting all live nodes on startup. One plausible reasoning for it is that other node had a hb io delay at the same time. The live threshold currently is 2. That's as low as it can be. As we set it to that because we start the heartbeat on mount.

With global heartbeat we can afford to increase that timeout. The patch increases it for only global heartbeat and that too only for the first heartbeat region.

Makes sense?

Quoted 6 lines · Sunil Mushran

We have seen isolated cases (very few, I might add) of o2hb not detecting all live nodes on startup. One plausible reasoning for it is that other node had a hb io delay at the same time. The live threshold currently is 2. That's as low as it can be. As we set it to that because we start the heartbeat on mount.

Quoted 5 lines

With global heartbeat we can afford to increase that timeout. The patch increases it for only global heartbeat and that too only for the first heartbeat region.

Makes sense?

I think Mark wants this in the patch description. We won't find
this email a year from now ;-)

If o2hb finds unexpected values in the heartbeat slot, it prints a message "ERROR: Device "dm-6": another node is heartbeating in our slot!"

This message could be misleading. This patch adds two more messages to help users better diagnose the problem.

Signed-off-by Sunil Mushran <[email protected]>