Remove BKL from fs/locks.c
Subsystems (6)
Arnd Bergmann
NAKPATCH 1/1 Remove BKL from fs/locks.c
From: Matthew Wilcox <[email protected]>
For reference, this is the patch I use for testing the BKL removal tree, which does not really remove the BKL at all as long as it is still required by fs/locks.c.
Christoph already made it clear that it cannot go upstream like this and Stephen said he consequently didn't want it in -next either. It may still be material for -mm or -rt, or just anyone who wants to test a kernel with CONFIG_BKL disabled.
It is also available in the a branch of my git tree:git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl.git nasty
I've taken a patch originally written by Matthew Wilcox and ported it to the current version. Unfortunately, the change conflicts with the use of lockd, which still heavily uses the big kernel lock.
As a workaround, I've made the behaviour configurable, it either uses the BKL when it's enabled or a spinlock when the BKL (and consequently nfs and lockd) are disabled.
Original introduction from Willy:
I've been promising to do this for about seven years now.
It seems to work well enough, but I haven't run any serious stress tests on it. This implementation uses one spinlock to protect both lock lists and all the i_flock chains. It doesn't seem worth splitting up the locking any further.
I had to move one memory allocation out from under the file_lock_lock. I hope I got that logic right. I'm rather tempted to split out the find_conflict algorithm from that function into something that can be called separately for the FL_ACCESS case.
I also have to drop and reacquire the file_lock_lock around the call to cond_resched(). This was done automatically for us before by the special BKL semantics.
I had to change vfs_setlease() as it relied on the special BKL ability to recursively acquire the same lock. The internal caller now calls __vfs_setlease and the exported interface acquires and releases the file_lock_lock around calling __vfs_setlease.
--- fs/Kconfig | 4 ++ fs/locks.c | 116 +++++++++++++++++++++++++++++++++++++----------------------- 2 files changed, 76 insertions(+), 44 deletions(-)
fs/Kconfig +4 -0
@@ -55,6 +55,10 @@ config FILE_LOCKING for filesystems like NFS and for the flock() system call. Disabling this option saves about 11k. +config FLOCKS_SPINLOCK + bool + default FILE_LOCKING && !LOCK_KERNEL + source "fs/notify/Kconfig" source "fs/quota/Kconfig"
fs/locks.c +72 -44
311 lines not on this page · raw message
Signature · 1 line
1.7.1
Quoted 6 lines · Arnd Bergmann
Christoph already made it clear that it cannot go upstream like this and Stephen said he consequently didn't want it in -next either. It may still be material for -mm or -rt, or just anyone who wants to test a kernel with
CONFIG_BKLdisabled.
I think something like this would be acceptable with that whole config option removed.
That would not help your particular "test without BKL at all" case, but basically I would suggest something like the following:
- do the unconditional conversion to the new spinlock, so thatfs/locks.c is internally consistent and conceptually works without the BKL at all.
- remove all the EXPORT_SYMBOL()s
- create wrapper functions named xyzzy_bkl() (perhaps in a differentfs/bkl.c file or whatever - or maybe just in the same file that you do the bkl for CONFIG_BKL in the first place) that wrap the fs/locks.c symbols with the kernel lock (so now those wrappers will take _both_ the BKL and the real spinlock)
- fix up all modular users mindlessly to call the bkl version, and
ship it that way for one release
End result: anybody who uses the old exports will just not link at all, and all in-kernel users (ie lockd etc) will now very explicitly have that xyz_bkl() use.
After that one release, we can re-introduce the old names without the bkl, and let filesystems and other subsystems just decide that they don't want the bkl version one by one. So that first phase would be basically a no-op with an interface rename just so that the rest could then be done piece-meal.
Linus
Quoted 9 lines · On Tuesday 14 September 2010 22:20:22 Linus Torvalds wrote:
End result: anybody who uses the old exports will just not link at all, and all in-kernel users (ie lockd etc) will now very explicitly have that
xyz_bkl()use.After that one release, we can re-introduce the old names without the bkl, and let filesystems and other subsystems just decide that they don't want the bkl version one by one. So that first phase would be basically a no-op with an interface rename just so that the rest could then be done piece-meal.
Hmm, maybe I'm misunderstanding part of that plan, but I think it won't work because the BKL in there seems to protect all struct file_lock instances and they are passed around to other code, most importantly lockd. The file_locks themselves can be accessed both using sys_flock() etc locally and using afs/nfs/coda/... exported file systems from other systems.
AFAICT we could end up with lockd changing data under BKL while a sys_flock() accesses the same data under file_lock_lock, with no mutual exclusion between the two.
Arnd
Quoted 7 lines · Arnd Bergmann
Hmm, maybe I'm misunderstanding part of that plan, but I think it won't work because the BKL in there seems to protect all struct
file_lockinstances and they are passed around to other code, most importantly lockd. Thefile_locksthemselves can be accessed both usingsys_flock()etc locally and using afs/nfs/coda/... exported file systems from other systems.
Ahh, so lockd actually walks those things itself too, not just using the fs/locks.c interfaces. Ok, scratch that then.
Oh well. I guess there's no incremental way to do things sanely. And nobody has patches to fix those users, I guess.
Linus
Quoted 2 lines · On Tuesday 14 September 2010 22:53:31 Linus Torvalds wrote:
Oh well. I guess there's no incremental way to do things sanely. And nobody has patches to fix those users, I guess.
The only critical user is fs/lockd, I can easily handle the rest. When I talked to Bruce and Trond during LinuxCon, they told me that it should be possible to separate the bits of fs/lockd that lock against fs/locks.c and convert the former to use lock_flocks().
That would be enough to actually apply this patch without the nasty CONFIG_BKL check and with changes to the few other consumers of file locks. My original plan was to have my current patch in -next until that happens.
Arnd
Quoted 8 lines · Arnd Bergmann
Quoted 2 lines · On Tuesday 14 September 2010 22:53:31 Linus Torvalds wrote:
Oh well. I guess there's no incremental way to do things sanely. And nobody has patches to fix those users, I guess.
The only critical user is fs/lockd, I can easily handle the rest. When I talked to Bruce and Trond during LinuxCon, they told me that it should be possible to separate the bits of fs/lockd that lock against
fs/locks.cand convert the former to uselock_flocks().
The NFSv4 client is quite ready to just switch to using lock_flocks() now. There is nothing that depends on the magical properties of the BKL. I do also have someone working on BKL removal for the NLM/lockd client and am hoping that should be ready in a couple of weeks time.
The timeline for the server is up to Bruce, however.
Quoted 4 lines
That would be enough to actually apply this patch without the nasty
CONFIG_BKLcheck and with changes to the few other consumers of file locks. My original plan was to have my current patch in -next until that happens.
Would it be possible to at least merge a patch that defines lock_flocks() (even if it just is an alias for lock_kernel()), so that we can convert those bits of the lock recovery code that need to peek into the inode->i_flock list ASAP?
Cheers
Trond
Quoted 4 lines · Trond Myklebust
Would it be possible to at least merge a patch that defines
lock_flocks()(even if it just is an alias forlock_kernel()), so that we can convert those bits of the lock recovery code that need to peek into theinode->i_flocklist ASAP?
Fine with me.
Another alternative would be to keep a series of all these patches together, like
1. fs/locks.c: prepare for BKL removal
2. fs/lockd: use lock/flocks
3. fs/locks.c: lock_flocks using spinlock (trivial)
Feel free to take the patch below into a tree of yours when adding patches 2 and 3 on top, or send me that patch once it's done and I'll add both into my tree.
Linus, Christoph: Does the patch below look ok? I've removed the CONFIG_BKL stuff in it now and changed over everything besides lockd to use lock_flocks(). I'd like to put this into the -next queue (after some basic testing).
---
Subject: [PATCH] fs/locks.c: prepare for BKL removal
This prepares the removal of the big kernel lock from the file locking code. We still use the BKL as long as fs/lockd uses it, but flip the definition to a private spinlock as soon as that's done.
All users outside of fs/lockd get converted to use lock_flocks() instead of lock_kernel() where appropriate.
Based on an earlier patch from Matthew Wilcox.
---
fs/afs/flock.c | 5 +- fs/ceph/mds_client.c | 6 +- fs/locks.c | 118 ++++++++++++++++++++++++++++--------------------- fs/nfs/delegation.c | 10 ++-- fs/nfs/nfs4state.c | 10 ++-- fs/nfsd/nfs4state.c | 6 +- include/linux/fs.h | 10 ++++ 7 files changed, 95 insertions(+), 70 deletions(-)
fs/afs/flock.c +2 -3
@@ -9,7 +9,6 @@* 2 of the License, or (at your option) any later version. */ -#include <linux/smp_lock.h> #include "internal.h" #define AFS_LOCK_GRANTED 0
@@ -274,7 +273,7 @@ static int afs_do_setlk(struct file *file, struct file_lock *fl) type = (fl->fl_type ==F_RDLCK ) ?AFS_LOCK_READ :AFS_LOCK_WRITE ; -lock_kernel (); +lock_flocks ();/* make sure we've got a callback on this file and that our view of the * data version is up to date */
@@ -421,7 +420,7 @@ given_lock:afs_vnode_fetch_status (vnode,NULL , key); error: -unlock_kernel (); +unlock_flocks ();_leave (" = %d" , ret);return ret;
fs/ceph/mds_client.c +3 -3
@@ -3,7 +3,7 @@#include <linux/wait.h> #include <linux/slab.h> #include <linux/sched.h> -#include <linux/smp_lock.h> +#include <linux/fs.h> #include "mds_client.h" #include "mon_client.h"
@@ -2362,7 +2362,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, if (recon_state->flock) { int num_fcntl_locks, num_flock_locks; - lock_kernel(); + lock_flocks(); ceph_count_locks(inode, &num_fcntl_locks, &num_flock_locks); rec.v2.flock_len = (2*sizeof(u32) + (num_fcntl_locks+num_flock_locks) *
@@ -2373,7 +2373,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, err =ceph_encode_locks (inode, pagelist, num_fcntl_locks, num_flock_locks); -unlock_kernel (); +unlock_flocks (); } out_free:
fs/locks.c +67 -51
336 lines not on this page · raw message
fs/nfs/delegation.c +5 -5
@@ -71,20 +71,20 @@ static int nfs_delegation_claim_locks(struct nfs_open_context *ctx, struct nfs4_if (inode->i_flock ==NULL ) goto out; -/* Protect inode->i_flock using the BKL */ -lock_kernel (); +/* Protect inode->i_flock using the file locks lock */ +lock_flocks ();for (fl = inode->i_flock; fl !=NULL ; fl = fl->fl_next) {if (!(fl->fl_flags & (FL_POSIX |FL_FLOCK )))continue ;if (nfs_file_open_context (fl->fl_file) != ctx)continue ; -unlock_kernel (); +unlock_flocks (); status =nfs4_lock_delegation_recall (state, fl);if (status <0 ) goto out; -lock_kernel (); +lock_flocks (); } -unlock_kernel (); +unlock_flocks (); out:return status; }
fs/nfs/nfs4state.c +5 -5
@@ -40,7 +40,7 @@#include <linux/kernel.h> #include <linux/slab.h> -#include <linux/smp_lock.h> +#include <linux/fs.h> #include <linux/nfs_fs.h> #include <linux/nfs_idmap.h> #include <linux/kthread.h>
@@ -970,13 +970,13 @@ static int nfs4_reclaim_locks(struct nfs4_state *state, const struct nfs4_state_/* Guard against delegation returns and new lock/unlock calls */ down_write (&nfsi->rwsem);/* Protect inode->i_flock using the BKL */ -lock_kernel (); +lock_flocks ();for (fl = inode->i_flock; fl !=NULL ; fl = fl->fl_next) {if (!(fl->fl_flags & (FL_POSIX |FL_FLOCK )))continue ;if (nfs_file_open_context (fl->fl_file)->state != state)continue ; -unlock_kernel (); +unlock_flocks (); status = ops->recover_lock (state, fl);switch (status) {case 0 :
@@ -1003,9 +1003,9 @@ static int nfs4_reclaim_locks(struct nfs4_state *state, const struct nfs4_state_/* kill_proc(fl->fl_pid, SIGLOST, 1); */ status =0 ; } -lock_kernel (); +lock_flocks (); } -unlock_kernel (); +unlock_flocks (); out:up_write (&nfsi->rwsem);return status;
fs/nfsd/nfs4state.c +3 -3
@@ -33,7 +33,7 @@*/ #include <linux/file.h> -#include <linux/smp_lock.h> +#include <linux/fs.h> #include <linux/slab.h> #include <linux/namei.h> #include <linux/swap.h>
@@ -3895,7 +3895,7 @@ check_for_locks(struct nfs4_file *filp, struct nfs4_stateowner *lowner)struct inode *inode = filp->fi_inode;int status =0 ; -lock_kernel (); +lock_flocks ();for (flpp = &inode->i_flock; *flpp !=NULL ; flpp = &(*flpp)->fl_next) {if ((*flpp)->fl_owner == (fl_owner_t )lowner) { status =1 ;
@@ -3903,7 +3903,7 @@ check_for_locks(struct nfs4_file *filp, struct nfs4_stateowner *lowner) } } out: -unlock_kernel (); +unlock_flocks ();return status; }
include/linux/fs.h +10 -0
@@ -1127,6 +1127,8 @@ extern int vfs_setlease(struct file *, long, struct file_lock **);extern int lease_modify (struct file_lock **,int );extern int lock_may_read (struct inode *,loff_t start,unsigned long count);extern int lock_may_write (struct inode *,loff_t start,unsigned long count); +extern void lock_flocks (void ); +extern void unlock_flocks (void );#else /* !CONFIG_FILE_LOCKING */ static inline int fcntl_getlk (struct file *file,struct flock __user *user) {
@@ -1269,6 +1271,14 @@ static inline int lock_may_write(struct inode *inode, loff_t start, return 1; } +static inline void lock_flocks () +{ +} + +static inline void unlock_flocks () +{ +} +#endif /* !CONFIG_FILE_LOCKING */
Hi Arnd,
Quoted 7 lines · Arnd Bergmann
Quoted 4 lines · Trond Myklebust
Would it be possible to at least merge a patch that defines
lock_flocks()(even if it just is an alias forlock_kernel()), so that we can convert those bits of the lock recovery code that need to peek into theinode->i_flocklist ASAP?Fine with me.
That's my preference. See below:
Quoted 104 lines
Another alternative would be to keep a series of all these patches together, like
1.
fs/locks.c: prepare for BKL removal
2. fs/lockd: use lock/flocks
3.fs/locks.c:lock_flocksusing spinlock (trivial)Feel free to take the patch below into a tree of yours when adding patches 2 and 3 on top, or send me that patch once it's done and I'll add both into my tree.
Linus, Christoph: Does the patch below look ok? I've removed the
CONFIG_BKLstuff in it now and changed over everything besides lockd to uselock_flocks(). I'd like to put this into the -next queue (after some basic testing).
---
Subject: [PATCH]fs/locks.c: prepare for BKL removalThis prepares the removal of the big kernel lock from the file locking code. We still use the BKL as long as fs/lockd uses it, but flip the definition to a private spinlock as soon as that's done.
All users outside of fs/lockd get converted to use
lock_flocks()instead oflock_kernel()where appropriate.Based on an earlier patch from Matthew Wilcox.
Signed-off-by Arnd Bergmann <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Christoph Hellwig <[email protected]>Cc Trond Myklebust <[email protected]>Cc "J. Bruce Fields" <[email protected]>Cc Miklos Szeredi <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Ingo Molnar <[email protected]>Cc John Kacur <[email protected]>---
fs/afs/flock.c | 5 +- fs/ceph/mds_client.c | 6 +- fs/locks.c | 118 ++++++++++++++++++++++++++++--------------------- fs/nfs/delegation.c | 10 ++-- fs/nfs/nfs4state.c | 10 ++-- fs/nfsd/nfs4state.c | 6 +- include/linux/fs.h | 10 ++++ 7 files changed, 95 insertions(+), 70 deletions(-)fs/afs/flock.c +2
-3@@ -9,7 +9,6 @@* 2 of the License, or (at your option) any later version. */ -#include <linux/smp_lock.h> #include "internal.h" #define AFS_LOCK_GRANTED 0@@ -274,7 +273,7 @@ static int afs_do_setlk(struct file *file, struct file_lock *fl) type = (fl->fl_type ==F_RDLCK ) ?AFS_LOCK_READ :AFS_LOCK_WRITE ; -lock_kernel (); +lock_flocks ();/* make sure we've got a callback on this file and that our view of the * data version is up to date */ @@ -421,7 +420,7 @@ given_lock:afs_vnode_fetch_status (vnode,NULL , key); error: -unlock_kernel (); +unlock_flocks ();_leave (" = %d" , ret);return ret;fs/ceph/mds_client.c +2
-2@@ -3,7 +3,7 @@#include <linux/wait.h> #include <linux/slab.h> #include <linux/sched.h> -#include <linux/smp_lock.h> +#include <linux/fs.h> #include "mds_client.h" #include "mon_client.h" @@ -2362,7 +2362,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, if (recon_state->flock) { int num_fcntl_locks, num_flock_locks; - lock_kernel(); + lock_flocks(); ceph_count_locks(inode, &num_fcntl_locks, &num_flock_locks); rec.v2.flock_len = (2*sizeof(u32) + (num_fcntl_locks+num_flock_locks) *
fs/ceph/mds_client.c static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, · line 2377
Quoted 6 lines
fs/ceph/mds_client.c +1
-1@@ -2373,7 +2373,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, err =ceph_encode_locks (inode, pagelist, num_fcntl_locks, num_flock_locks); -unlock_kernel (); +unlock_flocks (); }
The Ceph code won't currently behave with lock_flocks() taking a spinlock. We're preparing a patch to fix that now. As long as there is a window between lock_flocks() being defined and the spinlock conversion, I can send the fix upstream then and avoid any breakage. Or send the patches your way to include in your tree, whatever you prefer!
Quoted 14 lines
fs/ceph/mds_client.c +0
-0out_free:fs/locks.c +4
-0@@ -143,6 +143,21 @@ int lease_break_time = 45;static LIST_HEAD (file_lock_list);static LIST_HEAD (blocked_list); +/* +* Protects the two list heads above, plus the inode->i_flock list +* FIXME: should use a spinlock, once fs/lockd uses lock_flocks(). +*/
fs/locks.c int lease_break_time = 45; · line 159
Quoted 10 lines
fs/locks.c +10
-0+void lock_flocks (void ) +{ +lock_kernel (); +} +EXPORT_SYMBOL (lock_flocks); + +void unlock_flocks (void ) +{ +unlock_kernel (); +}
EXPORT_SYMBOL(unlock_flocks); ?
Thanks!
sage
Quoted 485 lines
fs/locks.c +53
-51+static struct kmem_cache *filelock_cache__read_mostly ;/* Allocate an empty lock structure. */ @@ -511,9 +526,9 @@ static void __locks_delete_block(struct file_lock *waiter)*/ static void locks_delete_block (struct file_lock *waiter) { -lock_kernel (); +lock_flocks ();__locks_delete_block (waiter); -unlock_kernel (); +unlock_flocks (); }/* Insert waiter into blocker's block list. @@ -644,7 +659,7 @@ posix_test_lock(struct file *filp, struct file_lock *fl) {struct file_lock *cfl; -lock_kernel (); +lock_flocks ();for (cfl = filp->f_path.dentry->d_inode->i_flock; cfl; cfl = cfl->fl_next) {if (!IS_POSIX (cfl))continue ;@@ -657,7 +672,7 @@ posix_test_lock(struct file *filp, struct file_lock *fl) fl->fl_pid =pid_vnr (cfl->fl_nspid); }else fl->fl_type =F_UNLCK ; -unlock_kernel (); +unlock_flocks ();return ; }EXPORT_SYMBOL (posix_test_lock);@@ -730,18 +745,16 @@ static int flock_lock_file(struct file *filp, struct file_lock *request)int error =0 ;int found =0 ; -lock_kernel (); -if (request->fl_flags &FL_ACCESS ) - goto find_conflict; - -if (request->fl_type !=F_UNLCK ) { - error = -ENOMEM ; +if (!(request->fl_flags &FL_ACCESS ) && (request->fl_type !=F_UNLCK )) { new_fl =locks_alloc_lock (); -if (new_fl ==NULL ) - goto out; - error =0 ; +if (!new_fl) +return -ENOMEM ; } +lock_flocks (); +if (request->fl_flags &FL_ACCESS ) + goto find_conflict; +for_each_lock (inode, before) {struct file_lock *fl = *before;if (IS_POSIX (fl))@@ -767,8 +780,11 @@ static int flock_lock_file(struct file *filp, struct file_lock *request)* If a higher-priority process was blocked on the old file lock, * give it the opportunity to lock the file. */ -if (found) +if (found) { +unlock_flocks ();cond_resched (); +lock_flocks (); + } find_conflict:for_each_lock (inode, before) {@@ -794,7 +810,7 @@ find_conflict: error =0 ; out: -unlock_kernel (); +unlock_flocks ();if (new_fl)locks_free_lock (new_fl);return error;@@ -823,7 +839,7 @@ static int __posix_lock_file(struct inode *inode, struct file_lock *request, str new_fl2 =locks_alloc_lock (); } -lock_kernel (); +lock_flocks ();if (request->fl_type !=F_UNLCK ) {for_each_lock (inode, before) { fl = *before;@@ -991,7 +1007,7 @@ static int __posix_lock_file(struct inode *inode, struct file_lock *request, strlocks_wake_up_blocks (left); } out: -unlock_kernel (); +unlock_flocks ();/* * Free any unused locks. */ @@ -1066,14 +1082,14 @@ int locks_mandatory_locked(struct inode *inode)/* * Search the lock list for this inode for any POSIX locks. */ -lock_kernel (); +lock_flocks ();for (fl = inode->i_flock; fl !=NULL ; fl = fl->fl_next) {if (!IS_POSIX (fl))continue ;if (fl->fl_owner != owner)break ; } -unlock_kernel (); +unlock_flocks ();return fl ? -EAGAIN :0 ; }@@ -1186,7 +1202,7 @@ int __break_lease(struct inode *inode, unsigned int mode) new_fl =lease_alloc (NULL , want_write ?F_WRLCK :F_RDLCK ); -lock_kernel (); +lock_flocks ();time_out_leases (inode);@@ -1247,8 +1263,10 @@ restart: break_time++; }locks_insert_block (flock, new_fl); +unlock_flocks (); error =wait_event_interruptible_timeout (new_fl->fl_wait, !new_fl->fl_next, break_time); +lock_flocks ();__locks_delete_block (new_fl);if (error >=0 ) {if (error ==0 )@@ -1263,7 +1281,7 @@ restart: } out: -unlock_kernel (); +unlock_flocks ();if (!IS_ERR (new_fl))locks_free_lock (new_fl);return error;@@ -1319,7 +1337,7 @@ int fcntl_getlease(struct file *filp)struct file_lock *fl;int type =F_UNLCK ; -lock_kernel (); +lock_flocks ();time_out_leases (filp->f_path.dentry->d_inode);for (fl = filp->f_path.dentry->d_inode->i_flock; fl &&IS_LEASE (fl); fl = fl->fl_next) {@@ -1328,7 +1346,7 @@ int fcntl_getlease(struct file *filp)break ; } } -unlock_kernel (); +unlock_flocks ();return type; }@@ -1341,7 +1359,7 @@ int fcntl_getlease(struct file *filp)* The (input) flp->fl_lmops->fl_break function is required * by break_lease(). * -* Called with kernel lock held. +* Called with file_lock_lock held. */ int generic_setlease (struct file *filp,long arg,struct file_lock **flp) {@@ -1359,6 +1377,13 @@ int generic_setlease(struct file *filp, long arg, struct file_lock **flp)if (error)return error; +if (arg !=F_UNLCK ) { + new_fl =locks_alloc_lock (); +if (new_fl ==NULL ) +return -ENOMEM ; + } + +lock_flocks ();time_out_leases (inode);BUG_ON (!(*flp)->fl_lmops->fl_break);@@ -1366,11 +1391,6 @@ int generic_setlease(struct file *filp, long arg, struct file_lock **flp) lease = *flp;if (arg !=F_UNLCK ) { - error = -ENOMEM ; - new_fl =locks_alloc_lock (); -if (new_fl ==NULL ) - goto out; - error = -EAGAIN ;if ((arg ==F_RDLCK ) && (atomic_read (&inode->i_writecount) >0 )) goto out;@@ -1427,16 +1447,18 @@ int generic_setlease(struct file *filp, long arg, struct file_lock **flp)locks_insert_lock (before, new_fl); *flp = new_fl; +unlock_flocks ();return 0 ; out:if (new_fl !=NULL )locks_free_lock (new_fl); +unlock_flocks ();return error; }EXPORT_SYMBOL (generic_setlease); -/** +/** * vfs_setlease - sets a lease on an open file * @filp: file pointer * @arg: type of lease to obtain @@ -1465,16 +1487,10 @@ EXPORT_SYMBOL(generic_setlease);int vfs_setlease (struct file *filp,long arg,struct file_lock **lease) { -int error; - -lock_kernel ();if (filp->f_op && filp->f_op->setlease) - error = filp->f_op->setlease (filp, arg, lease); +return filp->f_op->setlease (filp, arg, lease);else - error =generic_setlease (filp, arg, lease); -unlock_kernel (); - -return error; +return generic_setlease (filp, arg, lease); }EXPORT_SYMBOL_GPL (vfs_setlease);@@ -1499,12 +1515,12 @@ int fcntl_setlease(unsigned int fd, struct file *filp, long arg)if (error)return error; -lock_kernel (); - error =vfs_setlease (filp, arg, &flp);if (error || arg ==F_UNLCK ) goto out_unlock; +lock_flocks (); + error =fasync_helper (fd, filp,1 , &flp->fl_fasync);if (error <0 ) {/* remove lease just inserted by setlease */ @@ -1516,7 +1532,7 @@ int fcntl_setlease(unsigned int fd, struct file *filp, long arg) error =__f_setown (filp,task_pid (current),PIDTYPE_PID ,0 ); out_unlock: -unlock_kernel (); +unlock_flocks ();return error; }@@ -2020,7 +2036,7 @@ void locks_remove_flock(struct file *filp) fl.fl_ops->fl_release_private (&fl); } -lock_kernel (); +lock_flocks (); before = &inode->i_flock;while ((fl = *before) !=NULL ) {@@ -2038,7 +2054,7 @@ void locks_remove_flock(struct file *filp) } before = &fl->fl_next; } -unlock_kernel (); +unlock_flocks (); }/** @@ -2053,12 +2069,12 @@ posix_unblock_lock(struct file *filp, struct file_lock *waiter) {int status =0 ; -lock_kernel (); +lock_flocks ();if (waiter->fl_next)__locks_delete_block (waiter);else status = -ENOENT ; -unlock_kernel (); +unlock_flocks ();return status; }@@ -2172,7 +2188,7 @@ static int locks_show(struct seq_file *f, void *v)static void *locks_start (struct seq_file *f,loff_t *pos) { -lock_kernel (); +lock_flocks (); f->private = (void *)1 ;return seq_list_start (&file_lock_list, *pos); }@@ -2184,7 +2200,7 @@ static void *locks_next(struct seq_file *f, void *v, loff_t *pos)static void locks_stop (struct seq_file *f,void *v) { -unlock_kernel (); +unlock_flocks (); }static const struct seq_operations locks_seq_operations = {@@ -2231,7 +2247,7 @@ int lock_may_read(struct inode *inode, loff_t start, unsigned long len) {struct file_lock *fl;int result =1 ; -lock_kernel (); +lock_flocks ();for (fl = inode->i_flock; fl !=NULL ; fl = fl->fl_next) {if (IS_POSIX (fl)) {if (fl->fl_type ==F_RDLCK )@@ -2248,7 +2264,7 @@ int lock_may_read(struct inode *inode, loff_t start, unsigned long len) result =0 ;break ; } -unlock_kernel (); +unlock_flocks ();return result; }@@ -2271,7 +2287,7 @@ int lock_may_write(struct inode *inode, loff_t start, unsigned long len) {struct file_lock *fl;int result =1 ; -lock_kernel (); +lock_flocks ();for (fl = inode->i_flock; fl !=NULL ; fl = fl->fl_next) {if (IS_POSIX (fl)) {if ((fl->fl_end < start) || (fl->fl_start > (start + len)))@@ -2286,7 +2302,7 @@ int lock_may_write(struct inode *inode, loff_t start, unsigned long len) result =0 ;break ; } -unlock_kernel (); +unlock_flocks ();return result; }fs/nfs/delegation.c +5
-5@@ -71,20 +71,20 @@ static int nfs_delegation_claim_locks(struct nfs_open_context *ctx, struct nfs4_if (inode->i_flock ==NULL ) goto out; -/* Protect inode->i_flock using the BKL */ -lock_kernel (); +/* Protect inode->i_flock using the file locks lock */ +lock_flocks ();for (fl = inode->i_flock; fl !=NULL ; fl = fl->fl_next) {if (!(fl->fl_flags & (FL_POSIX |FL_FLOCK )))continue ;if (nfs_file_open_context (fl->fl_file) != ctx)continue ; -unlock_kernel (); +unlock_flocks (); status =nfs4_lock_delegation_recall (state, fl);if (status <0 ) goto out; -lock_kernel (); +lock_flocks (); } -unlock_kernel (); +unlock_flocks (); out:return status; }fs/nfs/nfs4state.c +5
-5@@ -40,7 +40,7 @@#include <linux/kernel.h> #include <linux/slab.h> -#include <linux/smp_lock.h> +#include <linux/fs.h> #include <linux/nfs_fs.h> #include <linux/nfs_idmap.h> #include <linux/kthread.h> @@ -970,13 +970,13 @@ static int nfs4_reclaim_locks(struct nfs4_state *state, const struct nfs4_state_/* Guard against delegation returns and new lock/unlock calls */ down_write (&nfsi->rwsem);/* Protect inode->i_flock using the BKL */ -lock_kernel (); +lock_flocks ();for (fl = inode->i_flock; fl !=NULL ; fl = fl->fl_next) {if (!(fl->fl_flags & (FL_POSIX |FL_FLOCK )))continue ;if (nfs_file_open_context (fl->fl_file)->state != state)continue ; -unlock_kernel (); +unlock_flocks (); status = ops->recover_lock (state, fl);switch (status) {case 0 :@@ -1003,9 +1003,9 @@ static int nfs4_reclaim_locks(struct nfs4_state *state, const struct nfs4_state_/* kill_proc(fl->fl_pid, SIGLOST, 1); */ status =0 ; } -lock_kernel (); +lock_flocks (); } -unlock_kernel (); +unlock_flocks (); out:up_write (&nfsi->rwsem);return status;fs/nfsd/nfs4state.c +3
-3@@ -33,7 +33,7 @@*/ #include <linux/file.h> -#include <linux/smp_lock.h> +#include <linux/fs.h> #include <linux/slab.h> #include <linux/namei.h> #include <linux/swap.h> @@ -3895,7 +3895,7 @@ check_for_locks(struct nfs4_file *filp, struct nfs4_stateowner *lowner)struct inode *inode = filp->fi_inode;int status =0 ; -lock_kernel (); +lock_flocks ();for (flpp = &inode->i_flock; *flpp !=NULL ; flpp = &(*flpp)->fl_next) {if ((*flpp)->fl_owner == (fl_owner_t )lowner) { status =1 ;@@ -3903,7 +3903,7 @@ check_for_locks(struct nfs4_file *filp, struct nfs4_stateowner *lowner) } } out: -unlock_kernel (); +unlock_flocks ();return status; }include/linux/fs.h +10
-1@@ -1127,6 +1127,8 @@ extern int vfs_setlease(struct file *, long, struct file_lock **);extern int lease_modify (struct file_lock **,int );extern int lock_may_read (struct inode *,loff_t start,unsigned long count);extern int lock_may_write (struct inode *,loff_t start,unsigned long count); +extern void lock_flocks (void ); +extern void unlock_flocks (void );#else /* !CONFIG_FILE_LOCKING */ static inline int fcntl_getlk (struct file *file,struct flock __user *user) {@@ -1269,6 +1271,14 @@ static inline int lock_may_write(struct inode *inode, loff_t start, return 1; } +static inline void lock_flocks () +{ +} + +static inline void unlock_flocks () +{ +} +#endif /* !CONFIG_FILE_LOCKING */ --To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 14 lines · Sage Weil
Quoted 14 lines
fs/ceph/mds_client.c +1
-2@@ -2362,7 +2362,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, if (recon_state->flock) { int num_fcntl_locks, num_flock_locks; - lock_kernel(); + lock_flocks(); ceph_count_locks(inode, &num_fcntl_locks, &num_flock_locks); rec.v2.flock_len = (2*sizeof(u32) + (num_fcntl_locks+num_flock_locks) *@@ -2373,7 +2373,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, err =ceph_encode_locks (inode, pagelist, num_fcntl_locks, num_flock_locks); -unlock_kernel ();
Quoted 8 lines
Review static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, · line 2377
Quoted 2 lines
(quoted diff) +1
-0+ unlock_flocks(); }The Ceph code won't currently behave with
lock_flocks()taking a spinlock. We're preparing a patch to fix that now. As long as there is a window betweenlock_flocks()being defined and the spinlock conversion, I can send the fix upstream then and avoid any breakage. Or send the patches your way to include in your tree, whatever you prefer!
I'd be happy to just integrate the fix in this patch, or as a separate patch in the series.
I certainly don't want to break any file system in the middle of the series, I'm sure we can find a way to do it right.
What is the problem? I just saw ceph_pagelist_addpage potentially sleeping, is that what you are thinking of?
Quoted 6 lines
fs/locks.c int lease_break_time = 45; · line 159
Quoted 4 lines
fs/locks.c +4
-0+void unlock_flocks (void ) +{ +unlock_kernel (); +}
EXPORT_SYMBOL(unlock_flocks); ?
Right, thanks!
Arnd
Quoted 24 lines · Arnd Bergmann
Quoted 14 lines · Sage Weil
Quoted 14 lines
fs/ceph/mds_client.c +1
-2@@ -2362,7 +2362,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, if (recon_state->flock) { int num_fcntl_locks, num_flock_locks; - lock_kernel(); + lock_flocks(); ceph_count_locks(inode, &num_fcntl_locks, &num_flock_locks); rec.v2.flock_len = (2*sizeof(u32) + (num_fcntl_locks+num_flock_locks) *@@ -2373,7 +2373,7 @@ static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, err =ceph_encode_locks (inode, pagelist, num_fcntl_locks, num_flock_locks); -unlock_kernel ();Quoted 8 lines
Review static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap, · line 2377
Quoted 2 lines
(quoted diff) +1
-0+ unlock_flocks(); }The Ceph code won't currently behave with
lock_flocks()taking a spinlock. We're preparing a patch to fix that now. As long as there is a window betweenlock_flocks()being defined and the spinlock conversion, I can send the fix upstream then and avoid any breakage. Or send the patches your way to include in your tree, whatever you prefer!
Quoted 8 lines
I'd be happy to just integrate the fix in this patch, or as a separate patch in the series.
I certainly don't want to break any file system in the middle of the series, I'm sure we can find a way to do it right.
What is the problem? I just saw
ceph_pagelist_addpagepotentially sleeping, is that what you are thinking of?
Yeah. The pagelist code needs to be adjusted to handle preallocation of potentially multiple pages. We'll send the fix your way shortly.
Thanks!
sage
Quoted 13 lines
Quoted 7 lines
fs/locks.c int lease_break_time = 45; · line 159
Quoted 5 lines
fs/locks.c +4
-0+void unlock_flocks (void ) +{ +unlock_kernel (); +}
EXPORT_SYMBOL(unlock_flocks); ?Right, thanks!
Arnd
Quoted 11 lines · Trond Myklebust
Quoted 8 lines · Arnd Bergmann
Quoted 2 lines · On Tuesday 14 September 2010 22:53:31 Linus Torvalds wrote:
Oh well. I guess there's no incremental way to do things sanely. And nobody has patches to fix those users, I guess.
The only critical user is fs/lockd, I can easily handle the rest. When I talked to Bruce and Trond during LinuxCon, they told me that it should be possible to separate the bits of fs/lockd that lock against
fs/locks.cand convert the former to uselock_flocks().
Quoted 6 lines
The NFSv4 client is quite ready to just switch to using
lock_flocks()now. There is nothing that depends on the magical properties of the BKL. I do also have someone working on BKL removal for the NLM/lockd client and am hoping that should be ready in a couple of weeks time.The timeline for the server is up to Bruce, however.
Looking over the server code.... The only code I see under the BKL is:
- a few lease callbacks in fs/nfsd/nfs4state.c, none of which sleep at this point, so all should be fine under a spinlock or whatever we want.
- fs/nfsd/nfs4state.c:check_for_locks(), which explicitly takes the BKL itself. All it does, though, is walk the lock list for a given file and check whether any of them have a given owner. It would be trivial to put it under some other lock and/or move it to locks.c.
--b.
Quoted 10 lines · On Wednesday 15 September 2010 22:42:03 J. Bruce Fields wrote:
Looking over the server code.... The only code I see under the BKL is:
- a few lease callbacks in
fs/nfsd/nfs4state.c, none of which sleep at this point, so all should be fine under a spinlock or whatever we want.
- fs/nfsd/nfs4state.c:check_for_locks(), which explicitly takes the BKL itself. All it does, though, is walk the lock list for a given file and check whether any of them have a given owner. It would be trivial to put it under some other lock and/or move it tolocks.c.
Ok. In the version of the patch I sent out yesterday, I came to the same conclusion and put both of these under lock_flocks(), which is still the BKL but can be converted to a spinlock after we have sorted out ceph and lockd. If you and others are fine with this patch, I'll add it to my bkl/config series.
Note that walking the i_flock list needs to use lock_flocks() (or the BKL), not a private lock. I guess that's what you are saying anyway, just making sure.
Arnd
Quoted 18 lines · Arnd Bergmann
Quoted 10 lines · On Wednesday 15 September 2010 22:42:03 J. Bruce Fields wrote:
Looking over the server code.... The only code I see under the BKL is:
- a few lease callbacks in
fs/nfsd/nfs4state.c, none of which sleep at this point, so all should be fine under a spinlock or whatever we want.
- fs/nfsd/nfs4state.c:check_for_locks(), which explicitly takes the BKL itself. All it does, though, is walk the lock list for a given file and check whether any of them have a given owner. It would be trivial to put it under some other lock and/or move it tolocks.c.Ok. In the version of the patch I sent out yesterday, I came to the same conclusion and put both of these under
lock_flocks(), which is still the BKL but can be converted to a spinlock after we have sorted out ceph and lockd. If you and others are fine with this patch, I'll add it to my bkl/config series.
Quoted 3 lines
Note that walking the
i_flocklist needs to uselock_flocks()(or the BKL), not a private lock. I guess that's what you are saying anyway, just making sure.
Yes, that's fine.
--b.