Skip to contentSkip to conversation

nfs locking for cluster filesystems

DISCUSSION J. Bruce Fields 2 participants 3 messages Apr 17, 2007

Subsystems (5)
lore
Share & tools
lore
#0 lore raw
Quoted 8 lines · J. Bruce Fields

If a filesystem defines a ->lock() method, that method is called when an application uses fcntl(), but it isn't called from the code path used by the lock managers (lockd (for nfsv2/v3) and nfsd itself (for v4)). The following patches remedy that. (This hasn't been a problem until recently because the filesystems that define ->lock (nfs, cifs, etc.) aren't filesystems that nfsd exports. The arrival of cluster filesystems changes that.)

Quoted 4 lines

We'd like an ack from Trond and/or Christoph before this goes into mainline, but it seems ready at least for testing in -mm.

The first four to six patches are actually just locks.c cleanup.

I've got an updated series addressing Christoph's comments (except that it's still using FL_CANCEL instead of a ->cancel() file method). I was arguing with myself about what would be the least obnoxious way to send the update:

- resend all 17? (Seems like mild overkill.)
- resend just the last 10? (The first 7 are identical to what's in -mm.)
- Send in an incremental patch? (But I really wouldn't want them submitted to the kernel that way.)
- Ask you to fetch from the server-cluster-locking-api branch of
git://linux-nfs.org/~bfields/linux.git
?

Preferences?

--b.

#1 lore raw
Quoted 20 lines · On Tue, 17 Apr 2007 13:55:33 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 8 lines · J. Bruce Fields

If a filesystem defines a ->lock() method, that method is called when an application uses fcntl(), but it isn't called from the code path used by the lock managers (lockd (for nfsv2/v3) and nfsd itself (for v4)). The following patches remedy that. (This hasn't been a problem until recently because the filesystems that define ->lock (nfs, cifs, etc.) aren't filesystems that nfsd exports. The arrival of cluster filesystems changes that.)

Quoted 4 lines

We'd like an ack from Trond and/or Christoph before this goes into mainline, but it seems ready at least for testing in -mm.

The first four to six patches are actually just locks.c cleanup.

I've got an updated series addressing Christoph's comments (except that it's still using FL_CANCEL instead of a ->cancel() file method). I was arguing with myself about what would be the least obnoxious way to send the update:

Quoted 5 lines

- resend all 17? (Seems like mild overkill.)
- resend just the last 10? (The first 7 are identical to what's in -mm.)
- Send in an incremental patch? (But I really wouldn't want them submitted to the kernel that way.)

Yes, there's no perfect approach here, if the changes are too large to permit the preferred approach of doing an incremental patch against each main patch.

Quoted 2 lines

- Ask you to fetch from the server-cluster-locking-api branch of
git://linux-nfs.org/~bfields/linux.git

Well if we're all now happy with the level of review then a git tree is OK by me. I'll suck that tree down later in the week, will let you know if anything goes wrong.

#2 lore raw
Quoted 12 lines · Andrew Morton
Quoted 11 lines · On Tue, 17 Apr 2007 13:55:33 -0400 "J. Bruce Fields" <[email protected]>…

I've got an updated series addressing Christoph's comments (except that it's still using FL_CANCEL instead of a ->cancel() file method). I was arguing with myself about what would be the least obnoxious way to send the update:

- resend all 17? (Seems like mild overkill.)
- resend just the last 10? (The first 7 are identical to what's in -mm.)
- Send in an incremental patch? (But I really wouldn't want them submitted to the kernel that way.)

Quoted 3 lines

Yes, there's no perfect approach here, if the changes are too large to permit the preferred approach of doing an incremental patch against each main patch.

The changes are small, but they touch several of the patches. And I reordered the patches a little too, just to add to the confusion.

Quoted 6 lines
Quoted 2 lines

- Ask you to fetch from the server-cluster-locking-api branch of
git://linux-nfs.org/~bfields/linux.git

Well if we're all now happy with the level of review then a git tree is OK by me. I'll suck that tree down later in the week, will let you know if anything goes wrong.

Thanks, that's obviously convenient for me. Christoph or Trond or anyone, if you'd rather have this in email, let me know.

I've appended the total diff between two versions just in case anyone's curious.

--b.

fs/gfs2/locking/dlm/plock.c +2 -2
index 1160cde..5207e4e 100644
@@ -89,8 +89,8 @@ int gdlm_plock(void *lockspace, struct lm_lockname *name,
 	op->info.start		= fl->fl_start;
 	op->info.end		= fl->fl_end;
 	op->info.owner		= (__u64)(long) fl->fl_owner;
-	if (fl->fl_lmops && fl->fl_lmops->fl_notify) {
-		xop->callback	= fl->fl_lmops->fl_notify;
+	if (fl->fl_lmops && fl->fl_lmops->fl_grant) {
+		xop->callback	= fl->fl_lmops->fl_grant;
 		/* might need to make a copy */
 		xop->fl		= fl;
 		xop->file	= file;
fs/lockd/svclock.c +31 -17

85 lines not on this page · raw message

fs/locks.c +4 -4
index 68ae4d4..13a3e02 100644
@@ -544,7 +544,7 @@ static void locks_wake_up_blocks(struct file_lock *blocker)
 				struct file_lock, fl_block);
 		__locks_delete_block(waiter);
 		if (waiter->fl_lmops && waiter->fl_lmops->fl_notify)
-			waiter->fl_lmops->fl_notify(waiter, NULL, -EAGAIN);
+			waiter->fl_lmops->fl_notify(waiter);
 		else
 			wake_up(&waiter->fl_wait);
 	}
@@ -1696,10 +1696,10 @@ out:
  * To avoid blocking kernel daemons, such as lockd, that need to acquire POSIX
  * locks, the ->lock() interface may return asynchronously, before the lock has
  * been granted or denied by the underlying filesystem, if (and only if)
- * fl_notify is set. Callers expecting ->lock() to return asynchronously
+ * fl_grant is set. Callers expecting ->lock() to return asynchronously
  * will only use F_SETLK, not F_SETLKW; they will set FL_SLEEP if (and only if)
  * the request is for a blocking lock. When ->lock() does return asynchronously,
- * it must return -EINPROGRESS, and call ->fl_notify() when the lock
+ * it must return -EINPROGRESS, and call ->fl_grant() when the lock
  * request completes.
  * If the request is for non-blocking lock the file system should return
  * -EINPROGRESS then try to get the lock and call the callback routine with
@@ -1709,7 +1709,7 @@ out:
  * grants a lock so the VFS can find out which locks are locally held and do
  * the correct lock cleanup when required.
  * The underlying filesystem must not drop the kernel lock or call
- * ->fl_notify() before returning to the caller with a -EINPROGRESS
+ * ->fl_grant() before returning to the caller with a -EINPROGRESS
  * return code.
  */
 int vfs_lock_file(struct file *filp, unsigned int cmd, struct file_lock *fl, struct file_lock *conf)
include/linux/fs.h +2 -1
index c58690d..b22991d 100644
@@ -784,7 +784,8 @@ struct file_lock_operations {
 
 struct lock_manager_operations {
 	int (*fl_compare_owner)(struct file_lock *, struct file_lock *);
-	int (*fl_notify)(struct file_lock *, struct file_lock *, int);
+	void (*fl_notify)(struct file_lock *);	/* unblock callback */
+	int (*fl_grant)(struct file_lock *, struct file_lock *, int);
 	void (*fl_copy_lock)(struct file_lock *, struct file_lock *);
 	void (*fl_release_private)(struct file_lock *);
 	void (*fl_break)(struct file_lock *);