Skip to contentSkip to conversation

vfs: dodge smp_mb in break_lease and break_deleg in the common case

PATCH Applied Mateusz Guzik 3 participants 3 messages Aug 6, 2024 – Aug 7, 2024 VFS

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vfs: dodge smp_mb in break_lease and break_deleg in the common case

These inlines show up in the fast path (e.g., in do_dentry_open()) and induce said full barrier regarding i_flctx access when in most cases the pointer is NULL.

The pointer can be safely checked before issuing the barrier, dodging it in most cases as a result.

It is plausible the consume fence would be sufficient, but I don't want to go audit all callers regarding what they before calling here.

---

the header file has locks_inode_context and i even found users like this (lease_get_mtime):

ctx = locks_inode_context(inode);
if (ctx && !list_empty_careful(&ctx->flc_lease)) {

however, without looking further at the code I'm not confident this would be sufficient here -- for all I know one consumer needs all stores to be visible before looking further after derefing the pointer

keeping the full fence in place makes this reasonably easy to reason about the change i think, but someone(tm) willing to sort this out is most welcome to do so

 include/linux/filelock.h | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
include/linux/filelock.h +12 -2
index daee999d05f3..bb44224c6676 100644
@@ -420,28 +420,38 @@ static inline int locks_lock_file_wait(struct file *filp, struct file_lock *fl)
 #ifdef CONFIG_FILE_LOCKING
 static inline int break_lease(struct inode *inode, unsigned int mode)
 {
+	struct file_lock_context *flctx;
+
 	/*
 	 * Since this check is lockless, we must ensure that any refcounts
 	 * taken are done before checking i_flctx->flc_lease. Otherwise, we
 	 * could end up racing with tasks trying to set a new lease on this
 	 * file.
 	 */
+	flctx = READ_ONCE(inode->i_flctx);
+	if (!flctx)
+		return 0;
 	smp_mb();
-	if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
+	if (!list_empty_careful(&flctx->flc_lease))
 		return __break_lease(inode, mode, FL_LEASE);
 	return 0;
 }
 
 static inline int break_deleg(struct inode *inode, unsigned int mode)
 {
+	struct file_lock_context *flctx;
+
 	/*
 	 * Since this check is lockless, we must ensure that any refcounts
 	 * taken are done before checking i_flctx->flc_lease. Otherwise, we
 	 * could end up racing with tasks trying to set a new lease on this
 	 * file.
 	 */
+	flctx = READ_ONCE(inode->i_flctx);
+	if (!flctx)
+		return 0;
 	smp_mb();
-	if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
+	if (!list_empty_careful(&flctx->flc_lease))
 		return __break_lease(inode, mode, FL_DELEG);
 	return 0;
 }
Signature · 1 line
2.43.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 19 lines · Mateusz Guzik

These inlines show up in the fast path (e.g., in do_dentry_open()) and induce said full barrier regarding i_flctx access when in most cases the pointer is NULL.

The pointer can be safely checked before issuing the barrier, dodging it in most cases as a result.

It is plausible the consume fence would be sufficient, but I don't want to go audit all callers regarding what they before calling here.

---

the header file has locks_inode_context and i even found users like this (lease_get_mtime):

ctx = locks_inode_context(inode);
if (ctx && !list_empty_careful(&ctx->flc_lease)) {

Quoted 8 lines

however, without looking further at the code I'm not confident this would be sufficient here -- for all I know one consumer needs all stores to be visible before looking further after derefing the pointer

keeping the full fence in place makes this reasonably easy to reason about the change i think, but someone(tm) willing to sort this out is most welcome to do so

Nod. It would be nice to get rid of that barrier. I'm not sure how to do that in a provably correct way. I'll need to think about that.

Quoted 45 lines

 include/linux/filelock.h | 14 ++++++++++++--

 1 file changed, 12 insertions(+), 2 deletions(-)
include/linux/filelock.h +6 -1
index daee999d05f3..bb44224c6676 100644
@@ -420,28 +420,38 @@ static inline int locks_lock_file_wait(struct file *filp, struct file_lock *fl)
  #ifdef CONFIG_FILE_LOCKING
  static inline int break_lease(struct inode *inode, unsigned int mode)
  {
+	struct file_lock_context *flctx;
+
  	/*
  	 * Since this check is lockless, we must ensure that any refcounts
  	 * taken are done before checking i_flctx->flc_lease. Otherwise, we
  	 * could end up racing with tasks trying to set a new lease on this
  	 * file.
  	 */
+	flctx = READ_ONCE(inode->i_flctx);
+	if (!flctx)
+		return 0;
  	smp_mb();
-	if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
+	if (!list_empty_careful(&flctx->flc_lease))
  		return __break_lease(inode, mode, FL_LEASE);
  	return 0;
  }


 static inline int break_deleg(struct inode *inode, unsigned int mode) {

include/linux/filelock.h +2 -0
+	struct file_lock_context *flctx;
+

  /*
  * Since this check is lockless, we must ensure that any refcounts
  * taken are done before checking i_flctx->flc_lease. Otherwise, we
  * could end up racing with tasks trying to set a new lease on this
  * file.
  */

include/linux/filelock.h +3 -0
+	flctx = READ_ONCE(inode->i_flctx);
+	if (!flctx)
+		return 0;

  smp_mb();

include/linux/filelock.h +1 -1
-	if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
+	if (!list_empty_careful(&flctx->flc_lease))
Quoted 3 lines

  return __break_lease(inode, mode, FL_DELEG); return 0;
 }

This change looks good to me:

maintainer Applied
#2 lore raw
In reply to Mateusz Guzik
Quoted 8 lines · Mateusz Guzik

These inlines show up in the fast path (e.g., in do_dentry_open()) and induce said full barrier regarding i_flctx access when in most cases the pointer is NULL.

The pointer can be safely checked before issuing the barrier, dodging it in most cases as a result.

[...]

Applied to the vfs.misc branch of the vfs/vfs.git tree. Patches in the vfs.misc branch should appear in linux-next soon.

Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.

It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.

Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.

tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs.misc

[1/1] vfs: dodge smp_mb in break_lease and break_deleg in the common case

      https://git.kernel.org/vfs/vfs/c/e23ee5db10d3