Skip to contentSkip to conversation

fs: dlm: plock, recovery and API deprecation

PATCH RESEND New Alexander Aring 1 participant 21 messages Jun 22, 2022

Subsystems (2)
lore
Share & tools
lore

Alexander Aring

author maintainer
#0 lore raw

cover fs: dlm: plock, recovery and API deprecation

Hi,

this patch series is just a resend to push them to dlm/next. It contains mainly three things:

1. Users asking while using plocks on gfs2 why they see "dev_write no op...". I reproduced a similar output by calling "stress-ng --fcntl 100" and hitting ctrl-c. However I think this message is harmless. The message occurs if the plock kernel handling cannot find the plock operation which was requested anymore as the user send a reply. Hitting ctrl-c at the right place we will delete the original requests from a lookup datastructure. First I added patches to signal that those original requests operations were removed and that's why a possible lookup will fail. However it confuses users, this patch will change the plock handling that we never should delete a request on purpose. If a interruption was happened during a requests to the user space and back we will signal this requests got interrupted and ignore the requests and cleanup the plock handling after we get a reply from the user space back. If there are still "dev_write no op..." messages it will signal a bug.

2. Changes due recovery handling e.g. wait until recovery is done when recovery is triggered by dlm_new_lockspace() and try to run the recovery mechanism several times if -EAGAIN is returned.

3. Remove the dlm waiters warning and deprecate the internal DLM timeout handling for now which will be removed in kernel v5.22. Upcomming patches to libdlm will effect that the linked application will disable the use of the feature which will be deprecated now.

- Alex

changes since on mailinglist:
- remove null terminated resource name in dlm traces
- add commit msg why it's safe to access rsb->res_name in traces
- move -EAGAIN in dlm_recover_directory() call (if -EINVAL is returned)
- remove patch "fs: dlm: add WARN_ON for non waiter case"
- remove patch "fs: dlm: add dlm_is_member() check if fenced"
- add "not" in comment about lkb IFL flags
- remove prototype of function which is removed in patch "fs: dlm: remove warn waiter handling"
- change prototype to static inline in patch
"fs: dlm: don't use deprecated API by default"

Alexander Aring (20):
fs: dlm: plock use list_first_entry
fs: dlm: remove may interrupted message
fs: dlm: add pid to debug log
fs: dlm: change log output to debug again
fs: dlm: use dlm_plock_info for do_unlock_close
fs: dlm: change posix lock sigint handling
fs: dlm: change ast and bast trace order
fs: dlm: remove additional dereference of lkbsb
fs: dlm: add resource name to tracepoints
fs: dlm: add notes for recovery and membership handling
fs: dlm: call dlm_lsop_recover_prep once
fs: dlm: let new_lockspace() wait until recovery
fs: dlm: handle recovery result outside of ls_recover
fs: dlm: handle recovery -EAGAIN case as retry
fs: dlm: change -EINVAL recovery error to -EAGAIN
fs: dlm: add comment about lkb IFL flags
fs: dlm: remove warn waiter handling
fs: dlm: remove timeout from dlm_user_adopt_orphan
fs: dlm: add API deprecation warning
fs: dlm: don't use deprecated API by default

 fs/dlm/Kconfig             |   9 +++
 fs/dlm/Makefile            |   2 +-
 fs/dlm/ast.c               |   4 +-
 fs/dlm/config.c            |  21 ++++--
 fs/dlm/config.h            |   3 +-
 fs/dlm/dir.c               |   4 ++
 fs/dlm/dlm_internal.h      |  32 +++++++--
 fs/dlm/lock.c              | 132 ++++++++++++++-----------------------
 fs/dlm/lock.h              |  17 ++++-
 fs/dlm/lockspace.c         |  31 +++++++--
 fs/dlm/member.c            |  30 ++++-----
 fs/dlm/netlink.c           |   8 +++
 fs/dlm/plock.c             |  51 +++++++++-----
 fs/dlm/recoverd.c          |  60 +++++++++++++++--
 fs/dlm/user.c              |  21 +++++-
 include/trace/events/dlm.h | 118 ++++++++++++++++++++++++++-------
 16 files changed, 378 insertions(+), 165 deletions(-)
Signature · 1 line
2.31.1
author maintainer
#1 lore raw

PATCH RESEND 1/20 fs: dlm: plock use list_first_entry

This patch will use the list helper list_first_entry() instead of using list_entry() to get the first element of a list.

---
 fs/dlm/plock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/dlm/plock.c +1 -1
index 0993eebf2060..7cab5d27132b 100644
@@ -378,7 +378,7 @@ static ssize_t dev_read(struct file *file, char __user *u, size_t count,
 
 	spin_lock(&ops_lock);
 	if (!list_empty(&send_list)) {
-		op = list_entry(send_list.next, struct plock_op, list);
+		op = list_first_entry(&send_list, struct plock_op, list);
 		if (op->info.flags & DLM_PLOCK_FL_CLOSE)
 			list_del(&op->list);
 		else
Signature · 1 line
2.31.1
author maintainer
#2 lore raw

PATCH RESEND 2/20 fs: dlm: remove may interrupted message

In reply to Alexander Aring

Commit bcfad4265ced ("dlm: improve plock logging if interrupted") shall improve the printout to give the user a reason why a dlm plock op function was not found anymore when the kernel dlm op side was sending a operation request to the user space.

However this situation was still confusing users about those messages. A new approach will avoid such message and if they occur there is a bug.

---
 fs/dlm/plock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/dlm/plock.c +1 -1
index 7cab5d27132b..32eda1f43d22 100644
@@ -443,7 +443,7 @@ static ssize_t dev_write(struct file *file, const char __user *u, size_t count,
 		else
 			wake_up(&recv_wq);
 	} else
-		log_print("%s: no op %x %llx - may got interrupted?", __func__,
+		log_print("%s: no op %x %llx", __func__,
 			  info.fsid, (unsigned long long)info.number);
 	return count;
 }
Signature · 1 line
2.31.1
author maintainer
#3 lore raw

PATCH RESEND 3/20 fs: dlm: add pid to debug log

In reply to Alexander Aring

This patch adds the pid information which requested the lock operation to the debug log output.

---
 fs/dlm/plock.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/dlm/plock.c +2 -2
index 32eda1f43d22..a58598c3edc5 100644
@@ -163,9 +163,9 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file,
 		spin_lock(&ops_lock);
 		list_del(&op->list);
 		spin_unlock(&ops_lock);
-		log_print("%s: wait interrupted %x %llx, op removed",
+		log_print("%s: wait interrupted %x %llx pid %d, op removed",
 			  __func__, ls->ls_global_id,
-			  (unsigned long long)number);
+			  (unsigned long long)number, op->info.pid);
 		dlm_release_plock_op(op);
 		do_unlock_close(ls, number, file, fl);
 		goto out;
Signature · 1 line
2.31.1
author maintainer
#4 lore raw

PATCH RESEND 4/20 fs: dlm: change log output to debug again

In reply to Alexander Aring

This patch revers the commit bcfad4265ced ("dlm: improve plock logging if interrupted") by moving it to debug level and noticing the user an op was removed from the kernel user space mechanism handling.

A new approach will be made that an op should never be removed when waiting from an answer from the user space side.

---
 fs/dlm/plock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/dlm/plock.c +1 -1
index a58598c3edc5..868940c48e3a 100644
@@ -163,7 +163,7 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file,
 		spin_lock(&ops_lock);
 		list_del(&op->list);
 		spin_unlock(&ops_lock);
-		log_print("%s: wait interrupted %x %llx pid %d, op removed",
+		log_debug(ls, "%s: wait interrupted %x %llx pid %d",
 			  __func__, ls->ls_global_id,
 			  (unsigned long long)number, op->info.pid);
 		dlm_release_plock_op(op);
Signature · 1 line
2.31.1
author maintainer
#5 lore raw

PATCH RESEND 5/20 fs: dlm: use dlm_plock_info for do_unlock_close

In reply to Alexander Aring

This patch will refactor do_unlock_close() by using only struct dlm_plock_info as parameter. There is only one use of do_unlock_close() and the same variables was copied before into the used dlm_plock_info structure.

---
 fs/dlm/plock.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)
fs/dlm/plock.c +6 -10
index 868940c48e3a..cf7bba461bfd 100644
@@ -79,8 +79,7 @@ static void send_op(struct plock_op *op)
    abandoned waiter.  So, we have to insert the unlock-close when the
    lock call is interrupted. */
 
-static void do_unlock_close(struct dlm_ls *ls, u64 number,
-			    struct file *file, struct file_lock *fl)
+static void do_unlock_close(const struct dlm_plock_info *info)
 {
 	struct plock_op *op;
 
@@ -89,15 +88,12 @@ static void do_unlock_close(struct dlm_ls *ls, u64 number,
 		return;
 
 	op->info.optype		= DLM_PLOCK_OP_UNLOCK;
-	op->info.pid		= fl->fl_pid;
-	op->info.fsid		= ls->ls_global_id;
-	op->info.number		= number;
+	op->info.pid		= info->pid;
+	op->info.fsid		= info->fsid;
+	op->info.number		= info->number;
 	op->info.start		= 0;
 	op->info.end		= OFFSET_MAX;
-	if (fl->fl_lmops && fl->fl_lmops->lm_grant)
-		op->info.owner	= (__u64) fl->fl_pid;
-	else
-		op->info.owner	= (__u64)(long) fl->fl_owner;
+	op->info.owner		= info->owner;
 
 	op->info.flags |= DLM_PLOCK_FL_CLOSE;
 	send_op(op);
@@ -167,7 +163,7 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file,
 			  __func__, ls->ls_global_id,
 			  (unsigned long long)number, op->info.pid);
 		dlm_release_plock_op(op);
-		do_unlock_close(ls, number, file, fl);
+		do_unlock_close(&op->info);
 		goto out;
 	}
 
Signature · 1 line
2.31.1
author maintainer
#6 lore raw

PATCH RESEND 6/20 fs: dlm: change posix lock sigint handling

In reply to Alexander Aring

This patch will change the handling if a plock operation was interrupted while waiting for a user space reply (probably dlm_controld). This is not while the posix lock waits in lock blocking state which is done by locks_lock_file_wait(). However the lock operation should be always interruptible, doesn't matter which wait is currently blocking the process.

If an interruption due waiting on a user space reply occurs the current behaviour is that we remove the already transmitted operation request to the user space from an list which is used to make a lookup if a reply comes back. This has as side effect that we see some:

dev_write no op...

in the kernel log because the lookup failed. This is easily reproducible by running:

stress-ng --fcntl 100

and hitting strg-c afterwards.

Instead of removing the op from the lookup list, we wait until the operation is completed. When the operation was completed we check if the wait was interrupted, if so we don't handle the request anymore and cleanup the original lock request. When there are still "dev_write no op" messages around it signals an issue that we removed an op while is hasn't been completed yet. This situation should never happen.

---
 fs/dlm/plock.c | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)
fs/dlm/plock.c +26 -3
index cf7bba461bfd..737f185aad8d 100644
@@ -29,6 +29,8 @@ struct plock_async_data {
 struct plock_op {
 	struct list_head list;
 	int done;
+	/* if lock op got interrupted while waiting dlm_controld reply */
+	bool sigint;
 	struct dlm_plock_info info;
 	/* if set indicates async handling */
 	struct plock_async_data *data;
@@ -157,16 +159,24 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file,
 	rv = wait_event_interruptible(recv_wq, (op->done != 0));
 	if (rv == -ERESTARTSYS) {
 		spin_lock(&ops_lock);
-		list_del(&op->list);
+		/* recheck under ops_lock if we got a done != 0,
+		 * if so this interrupt case should be ignored
+		 */
+		if (op->done != 0) {
+			spin_unlock(&ops_lock);
+			goto do_lock_wait;
+		}
+
+		op->sigint = true;
 		spin_unlock(&ops_lock);
 		log_debug(ls, "%s: wait interrupted %x %llx pid %d",
 			  __func__, ls->ls_global_id,
 			  (unsigned long long)number, op->info.pid);
-		dlm_release_plock_op(op);
-		do_unlock_close(&op->info);
 		goto out;
 	}
 
+do_lock_wait:
+
 	WARN_ON(!list_empty(&op->list));
 
 	rv = op->info.rv;
@@ -421,6 +431,19 @@ static ssize_t dev_write(struct file *file, const char __user *u, size_t count,
 		if (iter->info.fsid == info.fsid &&
 		    iter->info.number == info.number &&
 		    iter->info.owner == info.owner) {
+			if (iter->sigint) {
+				list_del(&iter->list);
+				spin_unlock(&ops_lock);
+
+				pr_debug("%s: sigint cleanup %x %llx pid %d",
+					  __func__, iter->info.fsid,
+					  (unsigned long long)iter->info.number,
+					  iter->info.pid);
+				do_unlock_close(&iter->info);
+				memcpy(&iter->info, &info, sizeof(info));
+				dlm_release_plock_op(iter);
+				return count;
+			}
 			list_del_init(&iter->list);
 			memcpy(&iter->info, &info, sizeof(info));
 			if (iter->data)
Signature · 1 line
2.31.1
author maintainer
#7 lore raw

PATCH RESEND 7/20 fs: dlm: change ast and bast trace order

In reply to Alexander Aring

This patch changes the order to call trace functionality before calling the traced callback. The intention is always to see at first that a dlm callback occurred and then optionally see dlm user traces in the ast or bast callback. Currently the behaviour is vice versa, the user sees that dlm ast or bast callback occurred after the dlm user callback for ast or bast was called.

---
 fs/dlm/ast.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/dlm/ast.c +2 -2
index bfac462dd3e8..df25c3e785cf 100644
@@ -255,13 +255,13 @@ void dlm_callback_work(struct work_struct *work)
 		if (callbacks[i].flags & DLM_CB_SKIP) {
 			continue;
 		} else if (callbacks[i].flags & DLM_CB_BAST) {
-			bastfn(lkb->lkb_astparam, callbacks[i].mode);
 			trace_dlm_bast(ls, lkb, callbacks[i].mode);
+			bastfn(lkb->lkb_astparam, callbacks[i].mode);
 		} else if (callbacks[i].flags & DLM_CB_CAST) {
 			lkb->lkb_lksb->sb_status = callbacks[i].sb_status;
 			lkb->lkb_lksb->sb_flags = callbacks[i].sb_flags;
-			castfn(lkb->lkb_astparam);
 			trace_dlm_ast(ls, lkb, lkb->lkb_lksb);
+			castfn(lkb->lkb_astparam);
 		}
 	}
 
Signature · 1 line
2.31.1
author maintainer
#8 lore raw

PATCH RESEND 8/20 fs: dlm: remove additional dereference of lkbsb

In reply to Alexander Aring

This patch removes a dereference of lkbsb of lkb when calling ast tracepoint. First it reduces additional overhead, even if traces are not acitivated. Second we can deference it in TP_fast_assign over the already passed lkb parameter.

---
 fs/dlm/ast.c               | 2 +-
 include/trace/events/dlm.h | 8 ++++----
 2 files changed, 5 insertions(+), 5 deletions(-)
fs/dlm/ast.c +1 -1
index df25c3e785cf..19ef136f9e4f 100644
@@ -260,7 +260,7 @@ void dlm_callback_work(struct work_struct *work)
 		} else if (callbacks[i].flags & DLM_CB_CAST) {
 			lkb->lkb_lksb->sb_status = callbacks[i].sb_status;
 			lkb->lkb_lksb->sb_flags = callbacks[i].sb_flags;
-			trace_dlm_ast(ls, lkb, lkb->lkb_lksb);
+			trace_dlm_ast(ls, lkb);
 			castfn(lkb->lkb_astparam);
 		}
 	}
include/trace/events/dlm.h +4 -4
index 32088c603244..e333176ecfaf 100644
@@ -138,9 +138,9 @@ TRACE_EVENT(dlm_bast,
 
 TRACE_EVENT(dlm_ast,
 
-	TP_PROTO(struct dlm_ls *ls, struct dlm_lkb *lkb, struct dlm_lksb *lksb),
+	TP_PROTO(struct dlm_ls *ls, struct dlm_lkb *lkb),
 
-	TP_ARGS(ls, lkb, lksb),
+	TP_ARGS(ls, lkb),
 
 	TP_STRUCT__entry(
 		__field(__u32, ls_id)
@@ -152,8 +152,8 @@ TRACE_EVENT(dlm_ast,
 	TP_fast_assign(
 		__entry->ls_id = ls->ls_global_id;
 		__entry->lkb_id = lkb->lkb_id;
-		__entry->sb_flags = lksb->sb_flags;
-		__entry->sb_status = lksb->sb_status;
+		__entry->sb_flags = lkb->lkb_lksb->sb_flags;
+		__entry->sb_status = lkb->lkb_lksb->sb_status;
 	),
 
 	TP_printk("ls_id=%u lkb_id=%x sb_flags=%s sb_status=%d",
Signature · 1 line
2.31.1
author maintainer
#9 lore raw

PATCH RESEND 9/20 fs: dlm: add resource name to tracepoints

In reply to Alexander Aring

This patch adds the resource name to dlm tracepoints. In case of dlm_lock() we might end in a short time situation (if flags is not convert) that a lkb is not associated with a resource at the time when the tracepoint is called. Therefore we have a a prioritzation of getting the resource name. If the resource in a lkb isn't set we use the name and namelen passed as parameter as fallback.

It should be okay to access the lkb_resource and the res_name field at the time when the tracepoint is invoked. The resource is assigned to a lkb and it's reference is being hold during the tracepoint call. During this time the resource cannot be freed. Also a lkb will never switch it's assigned resource. The name of a dlm_rsb is assigned at creation time and should never be changed during runtime as well.

Sometimes the resource name e.g. gfs2 is ascii encoded but this isn't required by DLM itself. If it's ascii encoded please note that the resource name is not null-terminated.

The TP_printk() call uses always a hexadecimal string array representation for the resource name.

---
 fs/dlm/lock.c              |   4 +-
 include/trace/events/dlm.h | 110 +++++++++++++++++++++++++++++++------
 2 files changed, 94 insertions(+), 20 deletions(-)
fs/dlm/lock.c +2 -2
index 226822f49d30..e80d42ba64ae 100644
@@ -3472,7 +3472,7 @@ int dlm_lock(dlm_lockspace_t *lockspace,
 	if (error)
 		goto out;
 
-	trace_dlm_lock_start(ls, lkb, mode, flags);
+	trace_dlm_lock_start(ls, lkb, name, namelen, mode, flags);
 
 	error = set_lock_args(mode, lksb, flags, namelen, 0, ast,
 			      astarg, bast, &args);
@@ -3487,7 +3487,7 @@ int dlm_lock(dlm_lockspace_t *lockspace,
 	if (error == -EINPROGRESS)
 		error = 0;
  out_put:
-	trace_dlm_lock_end(ls, lkb, mode, flags, error);
+	trace_dlm_lock_end(ls, lkb, name, namelen, mode, flags, error);
 
 	if (convert || error)
 		__put_lkb(ls, lkb);
include/trace/events/dlm.h +92 -18

225 lines not on this page · raw message

Signature · 1 line
2.31.1
author maintainer
#10 lore raw

PATCH RESEND 10/20 fs: dlm: add notes for recovery and membership handling

In reply to Alexander Aring

This patch adds some comment sections to make aware that the ls_recover() function should never fail before membership handling. Membership handling means to add/remove nodes from the lockspace ls_nodes attribute in dlm_recover_members().

This is because there are functionality like dlm_midcomms_add_member(), dlm_midcomms_remove_member() or dlm_lsop_recover_slot() which should always get aware of any join or leave of lockspace members. If we add a e.g. dlm_locking_stopped() before dlm_recover_members() to check if the recovery was interrupted and abort it we might skip to call dlm_midcomms_add_member(), dlm_midcomms_remove_member() or dlm_lsop_recover_slot().

A reason because the recovery is interrupted could be that the cluster manager notified about a new configuration .e.g. members joined or leaved. It is fine to interrupt or fail the recovery handling after the mentioned handling of dlm_recover_members() but never before.

---
 fs/dlm/member.c   | 6 +++++-
 fs/dlm/recoverd.c | 4 ++++
 2 files changed, 9 insertions(+), 1 deletion(-)
fs/dlm/member.c +5 -1
index 98084e0cfccf..7e5f5aefefb5 100644
@@ -534,7 +534,11 @@ int dlm_recover_members(struct dlm_ls *ls, struct dlm_recover *rv, int *neg_out)
 	int i, error, neg = 0, low = -1;
 
 	/* previously removed members that we've not finished removing need to
-	   count as a negative change so the "neg" recovery steps will happen */
+	 * count as a negative change so the "neg" recovery steps will happen
+	 *
+	 * This functionality must report all member changes to lsops or
+	 * midcomms layer and must never return before.
+	 */
 
 	list_for_each_entry(memb, &ls->ls_nodes_gone, list) {
 		log_rinfo(ls, "prev removed member %d", memb->nodeid);
fs/dlm/recoverd.c +4 -0
index a55dfce705dd..b5b519cde20b 100644
@@ -70,6 +70,10 @@ static int ls_recover(struct dlm_ls *ls, struct dlm_recover *rv)
 
 	/*
 	 * Add or remove nodes from the lockspace's ls_nodes list.
+	 *
+	 * Due the fact we must report all membership changes to lsops or
+	 * midcomms layer it is not permitted to abort ls_recover() until
+	 * this is done.
 	 */
 
 	error = dlm_recover_members(ls, rv, &neg);
Signature · 1 line
2.31.1
author maintainer
#11 lore raw

PATCH RESEND 11/20 fs: dlm: call dlm_lsop_recover_prep once

In reply to Alexander Aring

This patch changes the behaviour of "dlm_lsop_recover_prep" callback. It will be called once when locking was stopped if it was previously running not if dlm_ls_stop() is called multiple times when locking was already stopped. Now it will be called only if a dlm_ls_start() was before.

---
 fs/dlm/member.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
fs/dlm/member.c +10 -1
index 7e5f5aefefb5..67b056634f03 100644
@@ -679,7 +679,16 @@ int dlm_ls_stop(struct dlm_ls *ls)
 	if (!ls->ls_recover_begin)
 		ls->ls_recover_begin = jiffies;
 
-	dlm_lsop_recover_prep(ls);
+	/* call recover_prep ops only once and not multiple times
+	 * for each possible dlm_ls_stop() when recovery is already
+	 * stopped.
+	 *
+	 * If we successful was able to clear LSFL_RUNNING bit and
+	 * it was set we know it is the first dlm_ls_stop() call.
+	 */
+	if (new)
+		dlm_lsop_recover_prep(ls);
+
 	return 0;
 }
 
Signature · 1 line
2.31.1
author maintainer
#12 lore raw

PATCH RESEND 12/20 fs: dlm: let new_lockspace() wait until recovery

In reply to Alexander Aring

This patch changes the behaviour in dlm_new_lockspace() function to wait until a recovery was successful or failed. Before a possible waiter in ls_members_done was waiting until dlm_recover_members() was done either if it was successful (inclusive interrupted) or failed. The result was returned to the waiter of dlm_new_lockspace(), if success the caller was able to use the lockspace at this point.

This behaviour is now changed to wait of a complete run of recovery functionality which is done by ls_recover(). The result can be either successful or failed and delivered back to a possible waiter of ls_recovery_done. A possible waiter is then able to use the lockspace or run error handling if failed. If recovery gets interrupted e.g. checked at several places if dlm_locking_stopped() is true, a possible waiter of ls_recovery_done is still waiting until ls_recover() is successful or fails.

A reason why the recovery task gets interrupted is that an another dlm_ls_stop() was called while ls_recover() runs. The call of an another dlm_ls_stop() means that the recovery task will call ls_recover() again with a possible new configuration delivered by the cluster manager.

Most dlm kernel users e.g. gfs2 or cluster-md have their own wait handling to wait for recovery done after calling dlm_new_lockspace(). This becomes unnecessary now but still works. Users can update their code because dlm takes care about it now.

An example to simple interrupt recovery can be done by calling dlm_new_lockspace() and dlm_release_lockspace() in a loop on several cluster nodes. This has the effect that the cluster manager will interrupt the recovery with new membership information over and over again.

---
 fs/dlm/dlm_internal.h |  4 ++--
 fs/dlm/lockspace.c    |  9 +++++----
 fs/dlm/member.c       | 13 -------------
 fs/dlm/recoverd.c     | 13 +++++++++++++
 4 files changed, 20 insertions(+), 19 deletions(-)
fs/dlm/dlm_internal.h +2 -2
index 776c3ed519f0..c03388a3875c 100644
@@ -606,8 +606,8 @@ struct dlm_ls {
 
 	wait_queue_head_t	ls_uevent_wait;	/* user part of join/leave */
 	int			ls_uevent_result;
-	struct completion	ls_members_done;
-	int			ls_members_result;
+	struct completion	ls_recovery_done;
+	int			ls_recovery_result;
 
 	struct miscdevice       ls_device;
 
fs/dlm/lockspace.c +5 -4
index 19ed41a5da93..0c3613d09c5e 100644
@@ -548,8 +548,8 @@ static int new_lockspace(const char *name, const char *cluster,
 
 	init_waitqueue_head(&ls->ls_uevent_wait);
 	ls->ls_uevent_result = 0;
-	init_completion(&ls->ls_members_done);
-	ls->ls_members_result = -1;
+	init_completion(&ls->ls_recovery_done);
+	ls->ls_recovery_result = -1;
 
 	mutex_init(&ls->ls_cb_mutex);
 	INIT_LIST_HEAD(&ls->ls_cb_delay);
@@ -645,8 +645,9 @@ static int new_lockspace(const char *name, const char *cluster,
 	if (error)
 		goto out_recoverd;
 
-	wait_for_completion(&ls->ls_members_done);
-	error = ls->ls_members_result;
+	/* wait until recovery is successful or failed */
+	wait_for_completion(&ls->ls_recovery_done);
+	error = ls->ls_recovery_result;
 	if (error)
 		goto out_members;
 
fs/dlm/member.c +0 -13
index 67b056634f03..2af2ccfe43a9 100644
@@ -587,19 +587,6 @@ int dlm_recover_members(struct dlm_ls *ls, struct dlm_recover *rv, int *neg_out)
 	*neg_out = neg;
 
 	error = ping_members(ls);
-	/* error -EINTR means that a new recovery action is triggered.
-	 * We ignore this recovery action and let run the new one which might
-	 * have new member configuration.
-	 */
-	if (error == -EINTR)
-		error = 0;
-
-	/* new_lockspace() may be waiting to know if the config
-	 * is good or bad
-	 */
-	ls->ls_members_result = error;
-	complete(&ls->ls_members_done);
-
 	log_rinfo(ls, "dlm_recover_members %d nodes", ls->ls_num_nodes);
 	return error;
 }
fs/dlm/recoverd.c +13 -0
index b5b519cde20b..98c17f74927f 100644
@@ -243,6 +243,9 @@ static int ls_recover(struct dlm_ls *ls, struct dlm_recover *rv)
 		  jiffies_to_msecs(jiffies - start));
 	mutex_unlock(&ls->ls_recoverd_active);
 
+	ls->ls_recovery_result = 0;
+	complete(&ls->ls_recovery_done);
+
 	dlm_lsop_recover_done(ls);
 	return 0;
 
@@ -251,6 +254,16 @@ static int ls_recover(struct dlm_ls *ls, struct dlm_recover *rv)
 	log_rinfo(ls, "dlm_recover %llu error %d",
 		  (unsigned long long)rv->seq, error);
 	mutex_unlock(&ls->ls_recoverd_active);
+
+	/* let new_lockspace() get aware of critical error if recovery
+	 * was interrupted -EINTR we wait for the next ls_recover()
+	 * iteration until it succeeds.
+	 */
+	if (error != -EINTR) {
+		ls->ls_recovery_result = error;
+		complete(&ls->ls_recovery_done);
+	}
+
 	return error;
 }
 
Signature · 1 line
2.31.1
author maintainer
#13 lore raw

PATCH RESEND 13/20 fs: dlm: handle recovery result outside of ls_recover

In reply to Alexander Aring

This patch cleanups the recovery result handling by moving it outside of ls_recover() to the caller function do_ls_recovery(). This way it's clear how we react to recovery if it's successful or delivers different error codes.

---
 fs/dlm/recoverd.c | 42 ++++++++++++++++++++++++++----------------
 1 file changed, 26 insertions(+), 16 deletions(-)
fs/dlm/recoverd.c +26 -16
index 98c17f74927f..90e8b7f440da 100644
@@ -243,27 +243,12 @@ static int ls_recover(struct dlm_ls *ls, struct dlm_recover *rv)
 		  jiffies_to_msecs(jiffies - start));
 	mutex_unlock(&ls->ls_recoverd_active);
 
-	ls->ls_recovery_result = 0;
-	complete(&ls->ls_recovery_done);
-
-	dlm_lsop_recover_done(ls);
 	return 0;
 
  fail:
 	dlm_release_root_list(ls);
-	log_rinfo(ls, "dlm_recover %llu error %d",
-		  (unsigned long long)rv->seq, error);
 	mutex_unlock(&ls->ls_recoverd_active);
 
-	/* let new_lockspace() get aware of critical error if recovery
-	 * was interrupted -EINTR we wait for the next ls_recover()
-	 * iteration until it succeeds.
-	 */
-	if (error != -EINTR) {
-		ls->ls_recovery_result = error;
-		complete(&ls->ls_recovery_done);
-	}
-
 	return error;
 }
 
@@ -274,6 +259,7 @@ static int ls_recover(struct dlm_ls *ls, struct dlm_recover *rv)
 static void do_ls_recovery(struct dlm_ls *ls)
 {
 	struct dlm_recover *rv = NULL;
+	int error;
 
 	spin_lock(&ls->ls_recover_lock);
 	rv = ls->ls_recover_args;
@@ -283,7 +269,31 @@ static void do_ls_recovery(struct dlm_ls *ls)
 	spin_unlock(&ls->ls_recover_lock);
 
 	if (rv) {
-		ls_recover(ls, rv);
+		error = ls_recover(ls, rv);
+		switch (error) {
+		case 0:
+			ls->ls_recovery_result = 0;
+			complete(&ls->ls_recovery_done);
+
+			dlm_lsop_recover_done(ls);
+			break;
+		case -EINTR:
+			/* if recovery was interrupted -EINTR we wait for the next
+			 * ls_recover() iteration until it hopefully succeeds.
+			 */
+			log_rinfo(ls, "%s %llu interrupted and should be queued to run again",
+				  __func__, (unsigned long long)rv->seq);
+			break;
+		default:
+			log_rinfo(ls, "%s %llu error %d", __func__,
+				  (unsigned long long)rv->seq, error);
+
+			/* let new_lockspace() get aware of critical error */
+			ls->ls_recovery_result = error;
+			complete(&ls->ls_recovery_done);
+			break;
+		}
+
 		kfree(rv->nodes);
 		kfree(rv);
 	}
Signature · 1 line
2.31.1
author maintainer
#14 lore raw

PATCH RESEND 14/20 fs: dlm: handle recovery -EAGAIN case as retry

In reply to Alexander Aring

This patch adds additional functionality if recovery returns -EAGAIN error code to not deliver this failure to the potential caller of dlm_new_lockspace(). If -EAGAIN is returned we try to run recovery again and hope with a additional schedule() it doesn't return -EAGAIN anymore. If a maximum amount is hit, we fence ourself by running panic().

---
 fs/dlm/recoverd.c | 71 ++++++++++++++++++++++++++++++++---------------
 1 file changed, 48 insertions(+), 23 deletions(-)
fs/dlm/recoverd.c +48 -23
index 90e8b7f440da..2bd3bbe53828 100644
@@ -20,6 +20,7 @@
 #include "requestqueue.h"
 #include "recoverd.h"
 
+#define DLM_RECOVERY_MAX_RETRIES 5
 
 /* If the start for which we're re-enabling locking (seq) has been superseded
    by a newer stop (ls_recover_seq), we need to leave locking disabled.
@@ -259,7 +260,7 @@ static int ls_recover(struct dlm_ls *ls, struct dlm_recover *rv)
 static void do_ls_recovery(struct dlm_ls *ls)
 {
 	struct dlm_recover *rv = NULL;
-	int error;
+	int error, count = 0;
 
 	spin_lock(&ls->ls_recover_lock);
 	rv = ls->ls_recover_args;
@@ -269,30 +270,54 @@ static void do_ls_recovery(struct dlm_ls *ls)
 	spin_unlock(&ls->ls_recover_lock);
 
 	if (rv) {
-		error = ls_recover(ls, rv);
-		switch (error) {
-		case 0:
-			ls->ls_recovery_result = 0;
-			complete(&ls->ls_recovery_done);
-
-			dlm_lsop_recover_done(ls);
-			break;
-		case -EINTR:
-			/* if recovery was interrupted -EINTR we wait for the next
-			 * ls_recover() iteration until it hopefully succeeds.
+		do {
+			/* we try DLM_MAX_RECOVERY_RETRIES times again to run
+			 * recovery, if any -EAGAIN is not resolved this
+			 * time we will let DLM_ASSERT() fence ourself.
 			 */
-			log_rinfo(ls, "%s %llu interrupted and should be queued to run again",
-				  __func__, (unsigned long long)rv->seq);
-			break;
-		default:
-			log_rinfo(ls, "%s %llu error %d", __func__,
-				  (unsigned long long)rv->seq, error);
+			DLM_ASSERT(count < DLM_RECOVERY_MAX_RETRIES,
+				   pr_err("%s %llu too many recovery retries %d\n",
+					  __func__, (unsigned long long)rv->seq,
+					  DLM_RECOVERY_MAX_RETRIES););
+
+			error = ls_recover(ls, rv);
+			switch (error) {
+			case 0:
+				ls->ls_recovery_result = 0;
+				complete(&ls->ls_recovery_done);
+
+				dlm_lsop_recover_done(ls);
+				break;
+			case -EINTR:
+				/* if recovery was interrupted -EINTR we wait for the next
+				 * ls_recover() iteration until it hopefully succeeds.
+				 */
+				log_rinfo(ls,
+					  "%s %llu interrupted and should be queued to run again",
+					  __func__, (unsigned long long)rv->seq);
+				break;
+			case -EAGAIN:
+				/* either API is returning -EAGAIN or some critical errors
+				 * returning -EAGAIN which let the recovery run again. There
+				 * is a schedule() between it in the hope that the error resolves
+				 * itself. If not the above DLM_ASSERT() will hit.
+				 */
+				log_rinfo(ls, "%s %llu recovery wants to run again",
+					  __func__, (unsigned long long)rv->seq);
+				schedule();
+				break;
+			default:
+				log_rinfo(ls, "%s %llu error %d", __func__,
+					  (unsigned long long)rv->seq, error);
 
-			/* let new_lockspace() get aware of critical error */
-			ls->ls_recovery_result = error;
-			complete(&ls->ls_recovery_done);
-			break;
-		}
+				/* let new_lockspace() get aware of critical error */
+				ls->ls_recovery_result = error;
+				complete(&ls->ls_recovery_done);
+				break;
+			}
+
+			count++;
+		} while (error == -EAGAIN);
 
 		kfree(rv->nodes);
 		kfree(rv);
Signature · 1 line
2.31.1
author maintainer
#15 lore raw

PATCH RESEND 15/20 fs: dlm: change -EINVAL recovery error to -EAGAIN

In reply to Alexander Aring

This patch changes a -EINVAL error for dlm_master_lookup() to -EAGAIN. It is a critical error which should not happened, if it happens there exists an issue. However we still track those issues inside the lock but if they happen we try to run recovery again if those issues will get resolved. If not recovery has a logic to fail this node after several retries.

---
 fs/dlm/dir.c | 4 ++++
 1 file changed, 4 insertions(+)
fs/dlm/dir.c +4 -0
index fb1981654bb2..75cf0d4b2fe4 100644
@@ -138,6 +138,10 @@ int dlm_recover_directory(struct dlm_ls *ls)
 				if (error) {
 					log_error(ls, "recover_dir lookup %d",
 						  error);
+					/* retry recovery and hope it's gone */
+					if (error == -EINVAL)
+						error = -EAGAIN;
+
 					goto out_free;
 				}
 
Signature · 1 line
2.31.1
author maintainer
#16 lore raw

PATCH RESEND 16/20 fs: dlm: add comment about lkb IFL flags

In reply to Alexander Aring

This patch adds comments about the difference between the lower 2 bytes of lkb flags and the 2 upper bytes of the lkb IFL flags. In short the upper 2 bytes will be handled as internal flags whereas the lower 2 bytes are part of the DLM protocol and are used to exchange messages.

---
 fs/dlm/dlm_internal.h | 8 ++++++++
 1 file changed, 8 insertions(+)
fs/dlm/dlm_internal.h +8 -0
index c03388a3875c..afec22b1a65f 100644
@@ -207,6 +207,14 @@ struct dlm_args {
 #define DLM_IFL_TIMEOUT_CANCEL	0x00800000
 #define DLM_IFL_DEADLOCK_CANCEL	0x01000000
 #define DLM_IFL_STUB_MS		0x02000000 /* magic number for m_flags */
+/* least significant 2 bytes are message changed, they are full transmitted
+ * but at receive side only the 2 bytes LSB will be set.
+ *
+ * Even wireshark dlm dissector does only evaluate the lower bytes and note
+ * that they may not be used on transceiver side, we assume the higher bytes
+ * are for internal use or reserved so long they are not parsed on receiver
+ * side.
+ */
 #define DLM_IFL_USER		0x00000001
 #define DLM_IFL_ORPHAN		0x00000002
 
Signature · 1 line
2.31.1
author maintainer
#17 lore raw

PATCH RESEND 17/20 fs: dlm: remove warn waiter handling

In reply to Alexander Aring

This patch removes a feature in DLM to warn about if we don't get an answer back if we send a DLM message out and we wait for a reply message. There are in my opinion two reasons to add such warning, first is to know there might be a communication error and a message drop due communication errors. Second the other side dropped the message on stack because some reason.

However the first reason the midcomms layer will notify us if a message was received otherwise midcomms debugfs can tell us if there are still pending messages around.

For the second reason we can check the other nodes debug messages to see if there was a drop on application layer. However I think the initial idea of this feature was to check for the first reason.

This patch also removes the configfs entry waitwarn_us which is not being set by official dlm user space software, we assume fail to setting this configfs entry will not end in an critical error.

---
 fs/dlm/config.c       |  7 ----
 fs/dlm/config.h       |  1 -
 fs/dlm/dlm_internal.h |  1 -
 fs/dlm/lock.c         | 80 -------------------------------------------
 fs/dlm/lock.h         |  1 -
 fs/dlm/lockspace.c    |  1 -
 6 files changed, 91 deletions(-)
fs/dlm/config.c +0 -7
index 42eee2783756..081fd201e3a8 100644
@@ -76,7 +76,6 @@ struct dlm_cluster {
 	unsigned int cl_protocol;
 	unsigned int cl_mark;
 	unsigned int cl_timewarn_cs;
-	unsigned int cl_waitwarn_us;
 	unsigned int cl_new_rsb_count;
 	unsigned int cl_recover_callbacks;
 	char cl_cluster_name[DLM_LOCKSPACE_LEN];
@@ -103,7 +102,6 @@ enum {
 	CLUSTER_ATTR_PROTOCOL,
 	CLUSTER_ATTR_MARK,
 	CLUSTER_ATTR_TIMEWARN_CS,
-	CLUSTER_ATTR_WAITWARN_US,
 	CLUSTER_ATTR_NEW_RSB_COUNT,
 	CLUSTER_ATTR_RECOVER_CALLBACKS,
 	CLUSTER_ATTR_CLUSTER_NAME,
@@ -225,7 +223,6 @@ CLUSTER_ATTR(log_info, NULL);
 CLUSTER_ATTR(protocol, dlm_check_protocol_and_dlm_running);
 CLUSTER_ATTR(mark, NULL);
 CLUSTER_ATTR(timewarn_cs, dlm_check_zero);
-CLUSTER_ATTR(waitwarn_us, NULL);
 CLUSTER_ATTR(new_rsb_count, NULL);
 CLUSTER_ATTR(recover_callbacks, NULL);
 
@@ -241,7 +238,6 @@ static struct configfs_attribute *cluster_attrs[] = {
 	[CLUSTER_ATTR_PROTOCOL] = &cluster_attr_protocol,
 	[CLUSTER_ATTR_MARK] = &cluster_attr_mark,
 	[CLUSTER_ATTR_TIMEWARN_CS] = &cluster_attr_timewarn_cs,
-	[CLUSTER_ATTR_WAITWARN_US] = &cluster_attr_waitwarn_us,
 	[CLUSTER_ATTR_NEW_RSB_COUNT] = &cluster_attr_new_rsb_count,
 	[CLUSTER_ATTR_RECOVER_CALLBACKS] = &cluster_attr_recover_callbacks,
 	[CLUSTER_ATTR_CLUSTER_NAME] = &cluster_attr_cluster_name,
@@ -433,7 +429,6 @@ static struct config_group *make_cluster(struct config_group *g,
 	cl->cl_log_info = dlm_config.ci_log_info;
 	cl->cl_protocol = dlm_config.ci_protocol;
 	cl->cl_timewarn_cs = dlm_config.ci_timewarn_cs;
-	cl->cl_waitwarn_us = dlm_config.ci_waitwarn_us;
 	cl->cl_new_rsb_count = dlm_config.ci_new_rsb_count;
 	cl->cl_recover_callbacks = dlm_config.ci_recover_callbacks;
 	memcpy(cl->cl_cluster_name, dlm_config.ci_cluster_name,
@@ -955,7 +950,6 @@ int dlm_our_addr(struct sockaddr_storage *addr, int num)
 #define DEFAULT_PROTOCOL           DLM_PROTO_TCP
 #define DEFAULT_MARK               0
 #define DEFAULT_TIMEWARN_CS      500 /* 5 sec = 500 centiseconds */
-#define DEFAULT_WAITWARN_US	   0
 #define DEFAULT_NEW_RSB_COUNT    128
 #define DEFAULT_RECOVER_CALLBACKS  0
 #define DEFAULT_CLUSTER_NAME      ""
@@ -972,7 +966,6 @@ struct dlm_config_info dlm_config = {
 	.ci_protocol = DEFAULT_PROTOCOL,
 	.ci_mark = DEFAULT_MARK,
 	.ci_timewarn_cs = DEFAULT_TIMEWARN_CS,
-	.ci_waitwarn_us = DEFAULT_WAITWARN_US,
 	.ci_new_rsb_count = DEFAULT_NEW_RSB_COUNT,
 	.ci_recover_callbacks = DEFAULT_RECOVER_CALLBACKS,
 	.ci_cluster_name = DEFAULT_CLUSTER_NAME
fs/dlm/config.h +0 -1
index df92b0a07fc6..cb23d018e863 100644
@@ -38,7 +38,6 @@ struct dlm_config_info {
 	int ci_protocol;
 	int ci_mark;
 	int ci_timewarn_cs;
-	int ci_waitwarn_us;
 	int ci_new_rsb_count;
 	int ci_recover_callbacks;
 	char ci_cluster_name[DLM_LOCKSPACE_LEN];
fs/dlm/dlm_internal.h +0 -1
index afec22b1a65f..84dad619081e 100644
@@ -259,7 +259,6 @@ struct dlm_lkb {
 	struct list_head	lkb_ownqueue;	/* list of locks for a process */
 	struct list_head	lkb_time_list;
 	ktime_t			lkb_timestamp;
-	ktime_t			lkb_wait_time;
 	unsigned long		lkb_timeout_cs;
 
 	struct mutex		lkb_cb_mutex;
fs/dlm/lock.c +0 -80

98 lines not on this page · raw message

fs/dlm/lock.h +0 -1
index 252a5898f908..40781d9a24df 100644
@@ -24,7 +24,6 @@ int dlm_put_lkb(struct dlm_lkb *lkb);
 void dlm_scan_rsbs(struct dlm_ls *ls);
 int dlm_lock_recovery_try(struct dlm_ls *ls);
 void dlm_unlock_recovery(struct dlm_ls *ls);
-void dlm_scan_waiters(struct dlm_ls *ls);
 void dlm_scan_timeout(struct dlm_ls *ls);
 void dlm_adjust_timeouts(struct dlm_ls *ls);
 int dlm_master_lookup(struct dlm_ls *ls, int nodeid, char *name, int len,
fs/dlm/lockspace.c +0 -1
index 0c3613d09c5e..ca1eca0809d4 100644
@@ -275,7 +275,6 @@ static int dlm_scand(void *data)
 				ls->ls_scan_time = jiffies;
 				dlm_scan_rsbs(ls);
 				dlm_scan_timeout(ls);
-				dlm_scan_waiters(ls);
 				dlm_unlock_recovery(ls);
 			} else {
 				ls->ls_scan_time += HZ;
Signature · 1 line
2.31.1
author maintainer
#18 lore raw

PATCH RESEND 18/20 fs: dlm: remove timeout from dlm_user_adopt_orphan

In reply to Alexander Aring

This patch removes the timeout parameter from dlm_user_adopt_orphan(), it's never been used.

---
 fs/dlm/lock.c | 2 +-
 fs/dlm/lock.h | 2 +-
 fs/dlm/user.c | 1 -
 3 files changed, 2 insertions(+), 3 deletions(-)
fs/dlm/lock.c +1 -1
index 080cd216a9a4..12d4cc742308 100644
@@ -5886,7 +5886,7 @@ int dlm_user_convert(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
 
 int dlm_user_adopt_orphan(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
 		     int mode, uint32_t flags, void *name, unsigned int namelen,
-		     unsigned long timeout_cs, uint32_t *lkid)
+		     uint32_t *lkid)
 {
 	struct dlm_lkb *lkb = NULL, *iter;
 	struct dlm_user_args *ua;
fs/dlm/lock.h +1 -1
index 40781d9a24df..8c99e1b6eefa 100644
@@ -48,7 +48,7 @@ int dlm_user_convert(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
 	unsigned long timeout_cs);
 int dlm_user_adopt_orphan(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
 	int mode, uint32_t flags, void *name, unsigned int namelen,
-	unsigned long timeout_cs, uint32_t *lkid);
+	uint32_t *lkid);
 int dlm_user_unlock(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
 	uint32_t flags, uint32_t lkid, char *lvb_in);
 int dlm_user_cancel(struct dlm_ls *ls,  struct dlm_user_args *ua_tmp,
fs/dlm/user.c +0 -1
index 1060b24f18d4..4b2a24a6a15d 100644
@@ -270,7 +270,6 @@ static int device_user_lock(struct dlm_user_proc *proc,
 		error = dlm_user_adopt_orphan(ls, ua,
 					 params->mode, params->flags,
 					 params->name, params->namelen,
-					 (unsigned long) params->timeout,
 					 &lkid);
 		if (!error)
 			error = lkid;
Signature · 1 line
2.31.1
author maintainer
#19 lore raw

PATCH RESEND 19/20 fs: dlm: add API deprecation warning

In reply to Alexander Aring

This patch adds a deprecation config to DLM for API which will be removed in kernel v5.22. This deprecation warning will occur if somebody enables the deprecation API Kconfig entry and using the actual feature which will get deprecated as a notification to update their user space.

Note that the timeout feature for kernel locks was never been supported.

---
 fs/dlm/Kconfig     |  9 +++++++++
 fs/dlm/lockspace.c | 11 ++++++++++-
 fs/dlm/netlink.c   |  8 ++++++++
 fs/dlm/user.c      |  8 ++++++++
 4 files changed, 35 insertions(+), 1 deletion(-)
fs/dlm/Kconfig +9 -0
index ee92634196a8..cae58f21da08 100644
@@ -9,6 +9,15 @@ menuconfig DLM
 	A general purpose distributed lock manager for kernel or userspace
 	applications.
 
+config DLM_DEPRECATED_API
+	bool "DLM deprecated API"
+	depends on DLM
+	help
+	Enables DLM deprecated API which will be removed in later Linux
+	kernel releases.
+
+	If you are unsure, say N.
+
 config DLM_DEBUG
 	bool "DLM debugging"
 	depends on DLM
fs/dlm/lockspace.c +10 -1
index ca1eca0809d4..c9ec10700115 100644
@@ -489,8 +489,17 @@ static int new_lockspace(const char *name, const char *cluster,
 		ls->ls_ops_arg = ops_arg;
 	}
 
-	if (flags & DLM_LSFL_TIMEWARN)
+	if (flags & DLM_LSFL_TIMEWARN) {
+#ifdef CONFIG_DLM_DEPRECATED_API
+		pr_warn_once("===============================================================\n"
+			     "WARNING: the dlm DLM_LSFL_TIMEWARN flag is being deprecated and\n"
+			     "         will be removed in v5.22!\n"
+			     "         Inclusive DLM_LSFL_TIMEWARN define in UAPI header!\n"
+			     "===============================================================\n");
+#endif
+
 		set_bit(LSFL_TIMEWARN, &ls->ls_flags);
+	}
 
 	/* ls_exflags are forced to match among nodes, and we don't
 	   need to require all nodes to have some flags set */
fs/dlm/netlink.c +8 -0
index 67f68d48d60c..fed04860e550 100644
@@ -57,6 +57,14 @@ static int send_data(struct sk_buff *skb)
 
 static int user_cmd(struct sk_buff *skb, struct genl_info *info)
 {
+#ifdef CONFIG_DLM_DEPRECATED_API
+	pr_warn_once("====================================================\n"
+		     "WARNING: the dlm netlink API is being deprecated and\n"
+		     "         will be removed in v5.22!\n"
+		     "         Inclusive DLM netlink UAPI header!\n"
+		     "====================================================\n");
+#endif
+
 	listener_nlportid = info->snd_portid;
 	printk("user_cmd nlpid %u\n", listener_nlportid);
 	return 0;
fs/dlm/user.c +8 -0
index 4b2a24a6a15d..1fccb08bd825 100644
@@ -250,6 +250,14 @@ static int device_user_lock(struct dlm_user_proc *proc,
 		goto out;
 	}
 
+#ifdef CONFIG_DLM_DEPRECATED_API
+	if (params->timeout)
+		pr_warn_once("========================================================\n"
+			     "WARNING: the lkb timeout feature is being deprecated and\n"
+			     "         will be removed in v5.22!\n"
+			     "========================================================\n");
+#endif
+
 	ua = kzalloc(sizeof(struct dlm_user_args), GFP_NOFS);
 	if (!ua)
 		goto out;
Signature · 1 line
2.31.1
author maintainer
#20 lore raw

PATCH RESEND 20/20 fs: dlm: don't use deprecated API by default

In reply to Alexander Aring

This patch will disable the usage of current deprecated API if CONFIG_DLM_DEPRECATED_API is not set. This API will be removed in upcoming kernel release v5.22. I marked some places with a ifdef around deprecated API, at other places I used a no-op function replacement, depending on the situation how it's used and what was simpler to implement.

This patch disables the configfs entry timewarn_cs which will be set by official dlm user space software but does not end in a critical error. We assume it should be save to remove it.

---
 fs/dlm/Makefile       |  2 +-
 fs/dlm/config.c       | 14 +++++++++++++
 fs/dlm/config.h       |  2 ++
 fs/dlm/dlm_internal.h | 19 +++++++++++++++++-
 fs/dlm/lock.c         | 46 +++++++++++++++++++++++++++++++++++++++++++
 fs/dlm/lock.h         | 14 +++++++++++++
 fs/dlm/lockspace.c    | 14 ++++++++++---
 fs/dlm/user.c         | 12 +++++++++++
 8 files changed, 118 insertions(+), 5 deletions(-)
fs/dlm/Makefile +1 -1
index 3545fdafc6fb..71dab733cf9a 100644
@@ -9,7 +9,6 @@ dlm-y :=			ast.o \
 				member.o \
 				memory.o \
 				midcomms.o \
-				netlink.o \
 				lowcomms.o \
 				plock.o \
 				rcom.o \
@@ -18,5 +17,6 @@ dlm-y :=			ast.o \
 				requestqueue.o \
 				user.o \
 				util.o 
+dlm-$(CONFIG_DLM_DEPRECATED_API) +=	netlink.o
 dlm-$(CONFIG_DLM_DEBUG) +=	debug_fs.o
 
fs/dlm/config.c +14 -0
index 081fd201e3a8..ac8b62106ce0 100644
@@ -75,7 +75,9 @@ struct dlm_cluster {
 	unsigned int cl_log_info;
 	unsigned int cl_protocol;
 	unsigned int cl_mark;
+#ifdef CONFIG_DLM_DEPRECATED_API
 	unsigned int cl_timewarn_cs;
+#endif
 	unsigned int cl_new_rsb_count;
 	unsigned int cl_recover_callbacks;
 	char cl_cluster_name[DLM_LOCKSPACE_LEN];
@@ -101,7 +103,9 @@ enum {
 	CLUSTER_ATTR_LOG_INFO,
 	CLUSTER_ATTR_PROTOCOL,
 	CLUSTER_ATTR_MARK,
+#ifdef CONFIG_DLM_DEPRECATED_API
 	CLUSTER_ATTR_TIMEWARN_CS,
+#endif
 	CLUSTER_ATTR_NEW_RSB_COUNT,
 	CLUSTER_ATTR_RECOVER_CALLBACKS,
 	CLUSTER_ATTR_CLUSTER_NAME,
@@ -222,7 +226,9 @@ CLUSTER_ATTR(log_debug, NULL);
 CLUSTER_ATTR(log_info, NULL);
 CLUSTER_ATTR(protocol, dlm_check_protocol_and_dlm_running);
 CLUSTER_ATTR(mark, NULL);
+#ifdef CONFIG_DLM_DEPRECATED_API
 CLUSTER_ATTR(timewarn_cs, dlm_check_zero);
+#endif
 CLUSTER_ATTR(new_rsb_count, NULL);
 CLUSTER_ATTR(recover_callbacks, NULL);
 
@@ -237,7 +243,9 @@ static struct configfs_attribute *cluster_attrs[] = {
 	[CLUSTER_ATTR_LOG_INFO] = &cluster_attr_log_info,
 	[CLUSTER_ATTR_PROTOCOL] = &cluster_attr_protocol,
 	[CLUSTER_ATTR_MARK] = &cluster_attr_mark,
+#ifdef CONFIG_DLM_DEPRECATED_API
 	[CLUSTER_ATTR_TIMEWARN_CS] = &cluster_attr_timewarn_cs,
+#endif
 	[CLUSTER_ATTR_NEW_RSB_COUNT] = &cluster_attr_new_rsb_count,
 	[CLUSTER_ATTR_RECOVER_CALLBACKS] = &cluster_attr_recover_callbacks,
 	[CLUSTER_ATTR_CLUSTER_NAME] = &cluster_attr_cluster_name,
@@ -428,7 +436,9 @@ static struct config_group *make_cluster(struct config_group *g,
 	cl->cl_log_debug = dlm_config.ci_log_debug;
 	cl->cl_log_info = dlm_config.ci_log_info;
 	cl->cl_protocol = dlm_config.ci_protocol;
+#ifdef CONFIG_DLM_DEPRECATED_API
 	cl->cl_timewarn_cs = dlm_config.ci_timewarn_cs;
+#endif
 	cl->cl_new_rsb_count = dlm_config.ci_new_rsb_count;
 	cl->cl_recover_callbacks = dlm_config.ci_recover_callbacks;
 	memcpy(cl->cl_cluster_name, dlm_config.ci_cluster_name,
@@ -949,7 +959,9 @@ int dlm_our_addr(struct sockaddr_storage *addr, int num)
 #define DEFAULT_LOG_INFO           1
 #define DEFAULT_PROTOCOL           DLM_PROTO_TCP
 #define DEFAULT_MARK               0
+#ifdef CONFIG_DLM_DEPRECATED_API
 #define DEFAULT_TIMEWARN_CS      500 /* 5 sec = 500 centiseconds */
+#endif
 #define DEFAULT_NEW_RSB_COUNT    128
 #define DEFAULT_RECOVER_CALLBACKS  0
 #define DEFAULT_CLUSTER_NAME      ""
@@ -965,7 +977,9 @@ struct dlm_config_info dlm_config = {
 	.ci_log_info = DEFAULT_LOG_INFO,
 	.ci_protocol = DEFAULT_PROTOCOL,
 	.ci_mark = DEFAULT_MARK,
+#ifdef CONFIG_DLM_DEPRECATED_API
 	.ci_timewarn_cs = DEFAULT_TIMEWARN_CS,
+#endif
 	.ci_new_rsb_count = DEFAULT_NEW_RSB_COUNT,
 	.ci_recover_callbacks = DEFAULT_RECOVER_CALLBACKS,
 	.ci_cluster_name = DEFAULT_CLUSTER_NAME
fs/dlm/config.h +2 -0
index cb23d018e863..55c5f2c13ebd 100644
@@ -37,7 +37,9 @@ struct dlm_config_info {
 	int ci_log_info;
 	int ci_protocol;
 	int ci_mark;
+#ifdef CONFIG_DLM_DEPRECATED_API
 	int ci_timewarn_cs;
+#endif
 	int ci_new_rsb_count;
 	int ci_recover_callbacks;
 	char ci_cluster_name[DLM_LOCKSPACE_LEN];
fs/dlm/dlm_internal.h +18 -1
index 84dad619081e..8aca8085d24e 100644
@@ -145,7 +145,9 @@ struct dlm_args {
 	void			(*bastfn) (void *astparam, int mode);
 	int			mode;
 	struct dlm_lksb		*lksb;
+#ifdef CONFIG_DLM_DEPRECATED_API
 	unsigned long		timeout;
+#endif
 };
 
 
@@ -203,8 +205,10 @@ struct dlm_args {
 #define DLM_IFL_OVERLAP_UNLOCK  0x00080000
 #define DLM_IFL_OVERLAP_CANCEL  0x00100000
 #define DLM_IFL_ENDOFLIFE	0x00200000
+#ifdef CONFIG_DLM_DEPRECATED_API
 #define DLM_IFL_WATCH_TIMEWARN	0x00400000
 #define DLM_IFL_TIMEOUT_CANCEL	0x00800000
+#endif
 #define DLM_IFL_DEADLOCK_CANCEL	0x01000000
 #define DLM_IFL_STUB_MS		0x02000000 /* magic number for m_flags */
 /* least significant 2 bytes are message changed, they are full transmitted
@@ -257,9 +261,12 @@ struct dlm_lkb {
 	struct list_head	lkb_rsb_lookup;	/* waiting for rsb lookup */
 	struct list_head	lkb_wait_reply;	/* waiting for remote reply */
 	struct list_head	lkb_ownqueue;	/* list of locks for a process */
-	struct list_head	lkb_time_list;
 	ktime_t			lkb_timestamp;
+
+#ifdef CONFIG_DLM_DEPRECATED_API
+	struct list_head	lkb_time_list;
 	unsigned long		lkb_timeout_cs;
+#endif
 
 	struct mutex		lkb_cb_mutex;
 	struct work_struct	lkb_cb_work;
@@ -575,8 +582,10 @@ struct dlm_ls {
 	struct mutex		ls_orphans_mutex;
 	struct list_head	ls_orphans;
 
+#ifdef CONFIG_DLM_DEPRECATED_API
 	struct mutex		ls_timeout_mutex;
 	struct list_head	ls_timeout;
+#endif
 
 	spinlock_t		ls_new_rsb_spin;
 	int			ls_new_rsb_count;
@@ -695,7 +704,9 @@ struct dlm_ls {
 #define LSFL_RCOM_READY		5
 #define LSFL_RCOM_WAIT		6
 #define LSFL_UEVENT_WAIT	7
+#ifdef CONFIG_DLM_DEPRECATED_API
 #define LSFL_TIMEWARN		8
+#endif
 #define LSFL_CB_DELAY		9
 #define LSFL_NODIR		10
 
@@ -748,9 +759,15 @@ static inline int dlm_no_directory(struct dlm_ls *ls)
 	return test_bit(LSFL_NODIR, &ls->ls_flags);
 }
 
+#ifdef CONFIG_DLM_DEPRECATED_API
 int dlm_netlink_init(void);
 void dlm_netlink_exit(void);
 void dlm_timeout_warn(struct dlm_lkb *lkb);
+#else
+static inline int dlm_netlink_init(void) { return 0; }
+static inline void dlm_netlink_exit(void) { };
+static inline void dlm_timeout_warn(struct dlm_lkb *lkb) { };
+#endif
 int dlm_plock_init(void);
 void dlm_plock_exit(void);
 
fs/dlm/lock.c +46 -0

145 lines not on this page · raw message

fs/dlm/lock.h +14 -0
index 8c99e1b6eefa..a7b6474f009d 100644
@@ -24,8 +24,15 @@ int dlm_put_lkb(struct dlm_lkb *lkb);
 void dlm_scan_rsbs(struct dlm_ls *ls);
 int dlm_lock_recovery_try(struct dlm_ls *ls);
 void dlm_unlock_recovery(struct dlm_ls *ls);
+
+#ifdef CONFIG_DLM_DEPRECATED_API
 void dlm_scan_timeout(struct dlm_ls *ls);
 void dlm_adjust_timeouts(struct dlm_ls *ls);
+#else
+static inline void dlm_scan_timeout(struct dlm_ls *ls) { }
+static inline void dlm_adjust_timeouts(struct dlm_ls *ls) { }
+#endif
+
 int dlm_master_lookup(struct dlm_ls *ls, int nodeid, char *name, int len,
 		      unsigned int flags, int *r_nodeid, int *result);
 
@@ -40,12 +47,19 @@ void dlm_recover_waiters_pre(struct dlm_ls *ls);
 int dlm_recover_master_copy(struct dlm_ls *ls, struct dlm_rcom *rc);
 int dlm_recover_process_copy(struct dlm_ls *ls, struct dlm_rcom *rc);
 
+#ifdef CONFIG_DLM_DEPRECATED_API
 int dlm_user_request(struct dlm_ls *ls, struct dlm_user_args *ua, int mode,
 	uint32_t flags, void *name, unsigned int namelen,
 	unsigned long timeout_cs);
 int dlm_user_convert(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
 	int mode, uint32_t flags, uint32_t lkid, char *lvb_in,
 	unsigned long timeout_cs);
+#else
+int dlm_user_request(struct dlm_ls *ls, struct dlm_user_args *ua, int mode,
+	uint32_t flags, void *name, unsigned int namelen);
+int dlm_user_convert(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
+	int mode, uint32_t flags, uint32_t lkid, char *lvb_in);
+#endif
 int dlm_user_adopt_orphan(struct dlm_ls *ls, struct dlm_user_args *ua_tmp,
 	int mode, uint32_t flags, void *name, unsigned int namelen,
 	uint32_t *lkid);
fs/dlm/lockspace.c +11 -3
index c9ec10700115..6e449abdc5f4 100644
@@ -489,22 +489,28 @@ static int new_lockspace(const char *name, const char *cluster,
 		ls->ls_ops_arg = ops_arg;
 	}
 
-	if (flags & DLM_LSFL_TIMEWARN) {
 #ifdef CONFIG_DLM_DEPRECATED_API
+	if (flags & DLM_LSFL_TIMEWARN) {
 		pr_warn_once("===============================================================\n"
 			     "WARNING: the dlm DLM_LSFL_TIMEWARN flag is being deprecated and\n"
 			     "         will be removed in v5.22!\n"
 			     "         Inclusive DLM_LSFL_TIMEWARN define in UAPI header!\n"
 			     "===============================================================\n");
-#endif
 
 		set_bit(LSFL_TIMEWARN, &ls->ls_flags);
 	}
 
 	/* ls_exflags are forced to match among nodes, and we don't
-	   need to require all nodes to have some flags set */
+	 * need to require all nodes to have some flags set
+	 */
 	ls->ls_exflags = (flags & ~(DLM_LSFL_TIMEWARN | DLM_LSFL_FS |
 				    DLM_LSFL_NEWEXCL));
+#else
+	/* ls_exflags are forced to match among nodes, and we don't
+	 * need to require all nodes to have some flags set
+	 */
+	ls->ls_exflags = (flags & ~(DLM_LSFL_FS | DLM_LSFL_NEWEXCL));
+#endif
 
 	size = READ_ONCE(dlm_config.ci_rsbtbl_size);
 	ls->ls_rsbtbl_size = size;
@@ -535,8 +541,10 @@ static int new_lockspace(const char *name, const char *cluster,
 	mutex_init(&ls->ls_waiters_mutex);
 	INIT_LIST_HEAD(&ls->ls_orphans);
 	mutex_init(&ls->ls_orphans_mutex);
+#ifdef CONFIG_DLM_DEPRECATED_API
 	INIT_LIST_HEAD(&ls->ls_timeout);
 	mutex_init(&ls->ls_timeout_mutex);
+#endif
 
 	INIT_LIST_HEAD(&ls->ls_new_rsb);
 	spin_lock_init(&ls->ls_new_rsb_spin);
fs/dlm/user.c +12 -0
index 1fccb08bd825..999918348b31 100644
@@ -270,10 +270,16 @@ static int device_user_lock(struct dlm_user_proc *proc,
 	ua->xid = params->xid;
 
 	if (params->flags & DLM_LKF_CONVERT) {
+#ifdef CONFIG_DLM_DEPRECATED_API
 		error = dlm_user_convert(ls, ua,
 				         params->mode, params->flags,
 				         params->lkid, params->lvb,
 					 (unsigned long) params->timeout);
+#else
+		error = dlm_user_convert(ls, ua,
+					 params->mode, params->flags,
+					 params->lkid, params->lvb);
+#endif
 	} else if (params->flags & DLM_LKF_ORPHAN) {
 		error = dlm_user_adopt_orphan(ls, ua,
 					 params->mode, params->flags,
@@ -282,10 +288,16 @@ static int device_user_lock(struct dlm_user_proc *proc,
 		if (!error)
 			error = lkid;
 	} else {
+#ifdef CONFIG_DLM_DEPRECATED_API
 		error = dlm_user_request(ls, ua,
 					 params->mode, params->flags,
 					 params->name, params->namelen,
 					 (unsigned long) params->timeout);
+#else
+		error = dlm_user_request(ls, ua,
+					 params->mode, params->flags,
+					 params->name, params->namelen);
+#endif
 		if (!error)
 			error = ua->lksb.sb_lkid;
 	}
Signature · 1 line
2.31.1