Skip to contentSkip to conversation

overlayfs lock ordering changes

PATCH Superseded Amir Goldstein 2 participants 7 messages Jul 20, 2023 – Aug 14, 2023 Overlay filesystem

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Amir Goldstein

author maintainer
#0 lore raw

cover overlayfs lock ordering changes

Hi Miklos,

These prep patches are needed for my start-write-safe series [1]. This is not urgent and the prep patches don't need to be merged for next cycle, but I think these are good changes regardless, so wanted to post them for early review - if you like them you can queue them for 6.6.

It is quite hard to do the review of the locking reorder patch from the diff itself and I couldn't figure out a better way to split this change. I've intentionally left some otherwise useless out: goto labels to make the patch review a bit simper - they could be removed later.

On the good side, lockdep was very tough with me and it easily detected bugs in the earlier versions of the patches.

Going on vacation. will be back round rc6.

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/start-write-safe

Amir Goldstein (2):
ovl: reorder ovl_want_write() after ovl_inode_lock()
ovl: avoid lockdep warning with open and llseek of lower file

 fs/overlayfs/copy_up.c | 34 +++++++++++++-------
 fs/overlayfs/dir.c     | 71 ++++++++++++++++++------------------------
 fs/overlayfs/export.c  |  7 +----
 fs/overlayfs/inode.c   | 56 ++++++++++++++++-----------------
 fs/overlayfs/util.c    | 18 +++++++++--
 5 files changed, 97 insertions(+), 89 deletions(-)
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH 1/2 ovl: reorder ovl_want_write() after ovl_inode_lock()

Make the locking order of ovl_inode_lock() strictly between the two vfs stacked layers, i.e.:
- ovl vfs locks: sb_writers, inode_lock, ...
- ovl_inode_lock
- upper vfs locks: sb_writers, inode_lock, ...

To that effect, move ovl_want_write() into the helpers ovl_nlink_start() and ovl_copy_up_start() which currently take the ovl_inode_lock() after ovl_want_write().

---
 fs/overlayfs/copy_up.c | 13 ++------
 fs/overlayfs/dir.c     | 71 ++++++++++++++++++------------------------
 fs/overlayfs/export.c  |  7 +----
 fs/overlayfs/inode.c   | 56 ++++++++++++++++-----------------
 fs/overlayfs/util.c    | 18 +++++++++--
 5 files changed, 77 insertions(+), 88 deletions(-)
fs/overlayfs/copy_up.c +3 -10
index 568f743a5584..7e09c3187274 100644
@@ -1129,17 +1129,10 @@ static bool ovl_open_need_copy_up(struct dentry *dentry, int flags)
 
 int ovl_maybe_copy_up(struct dentry *dentry, int flags)
 {
-	int err = 0;
-
-	if (ovl_open_need_copy_up(dentry, flags)) {
-		err = ovl_want_write(dentry);
-		if (!err) {
-			err = ovl_copy_up_flags(dentry, flags);
-			ovl_drop_write(dentry);
-		}
-	}
+	if (!ovl_open_need_copy_up(dentry, flags))
+		return 0;
 
-	return err;
+	return ovl_copy_up_flags(dentry, flags);
 }
 
 int ovl_copy_up_with_data(struct dentry *dentry)
fs/overlayfs/dir.c +31 -40

167 lines not on this page · raw message

fs/overlayfs/export.c +1 -6
index 35680b6e175b..9e4881458c88 100644
@@ -23,12 +23,7 @@ static int ovl_encode_maybe_copy_up(struct dentry *dentry)
 	if (ovl_dentry_upper(dentry))
 		return 0;
 
-	err = ovl_want_write(dentry);
-	if (!err) {
-		err = ovl_copy_up(dentry);
-		ovl_drop_write(dentry);
-	}
-
+	err = ovl_copy_up(dentry);
 	if (err) {
 		pr_warn_ratelimited("failed to copy up on encode (%pd2, err=%i)\n",
 				    dentry, err);
fs/overlayfs/inode.c +26 -30

167 lines not on this page · raw message

fs/overlayfs/util.c +16 -2
index 7ef9e13c404a..cc1625d884d0 100644
@@ -649,16 +649,23 @@ int ovl_copy_up_start(struct dentry *dentry, int flags)
 	int err;
 
 	err = ovl_inode_lock_interruptible(inode);
-	if (!err && ovl_already_copied_up_locked(dentry, flags)) {
+	if (err)
+		return err;
+
+	if (ovl_already_copied_up_locked(dentry, flags))
 		err = 1; /* Already copied up */
+	else
+		err = ovl_want_write(dentry);
+
+	if (err)
 		ovl_inode_unlock(inode);
-	}
 
 	return err;
 }
 
 void ovl_copy_up_end(struct dentry *dentry)
 {
+	ovl_drop_write(dentry);
 	ovl_inode_unlock(d_inode(dentry));
 }
 
@@ -1000,6 +1007,10 @@ int ovl_nlink_start(struct dentry *dentry)
 	if (err)
 		return err;
 
+	err = ovl_want_write(dentry);
+	if (err)
+		goto out;
+
 	if (d_is_dir(dentry) || !ovl_test_flag(OVL_INDEX, inode))
 		goto out;
 
@@ -1012,6 +1023,8 @@ int ovl_nlink_start(struct dentry *dentry)
 	 */
 	err = ovl_set_nlink_upper(dentry);
 	revert_creds(old_cred);
+	if (err)
+		ovl_drop_write(dentry);
 
 out:
 	if (err)
@@ -1032,6 +1045,7 @@ void ovl_nlink_end(struct dentry *dentry)
 		revert_creds(old_cred);
 	}
 
+	ovl_drop_write(dentry);
 	ovl_inode_unlock(inode);
 }
 
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH 2/2 ovl: avoid lockdep warning with open and llseek of lower file

In reply to Amir Goldstein

overlayfs file open (ovl_maybe_lookup_lowerdata) and overlay file llseek take the ovl_inode_lock, without holding upper sb_writers.

In case of nested lower overlay that uses same upper fs as this overlay, lockdep will warn about (possibly false positive) circular lock dependency when doing open/llseek of lower ovl file during copy up with our upper sb_writers held, because the locking ordering seems reverse to the locking order in ovl_copy_up_start():

- lower ovl_inode_lock
- upper sb_writers

Drop upper sb_writers for lower file open and llseek to avoid the lockdep warning.

---
 fs/overlayfs/copy_up.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
fs/overlayfs/copy_up.c +20 -1
index 7e09c3187274..467107bb9f0e 100644
@@ -233,6 +233,7 @@ static int ovl_copy_fileattr(struct inode *inode, const struct path *old,
 static int ovl_copy_up_file(struct ovl_fs *ofs, struct dentry *dentry,
 			    struct file *new_file, loff_t len)
 {
+	struct super_block *upper_sb = ovl_upper_mnt(ofs)->mnt_sb;
 	struct path datapath;
 	struct file *old_file;
 	loff_t old_pos = 0;
@@ -247,9 +248,21 @@ static int ovl_copy_up_file(struct ovl_fs *ofs, struct dentry *dentry,
 	if (WARN_ON(datapath.dentry == NULL))
 		return -EIO;
 
+	/*
+	 * Drop upper sb_writers to avert lockdep warning with open of lower
+	 * file in nested overlay:
+	 * - upper sb_writers
+	 * -- lower ovl_inode_lock (ovl_maybe_lookup_lowerdata)
+	 *
+	 * In case lower ovl uses same upper as this ovl, lockdep will warn
+	 * about (possibly false positive) circular lock dependency because the
+	 * ordering above is reverse to the order in ovl_copy_up_start().
+	 */
+	sb_end_write(upper_sb);
 	old_file = ovl_path_open(&datapath, O_LARGEFILE | O_RDONLY);
 	if (IS_ERR(old_file))
 		return PTR_ERR(old_file);
+	sb_start_write(upper_sb);
 
 	/* Try to use clone_file_range to clone up within the same fs */
 	cloned = do_clone_file_range(old_file, 0, new_file, 0, len, 0);
@@ -287,10 +300,16 @@ static int ovl_copy_up_file(struct ovl_fs *ofs, struct dentry *dentry,
 		 * it may not recognize all kind of holes and sometimes
 		 * only skips partial of hole area. However, it will be
 		 * enough for most of the use cases.
+		 *
+		 * Drop upper sb_writers to avert lockdep warning with
+		 * llseek of lower file in nested overlay:
+		 * - upper sb_writers
+		 * -- lower ovl_inode_lock (ovl_llseek)
 		 */
-
 		if (skip_hole && data_pos < old_pos) {
+			sb_end_write(upper_sb);
 			data_pos = vfs_llseek(old_file, old_pos, SEEK_DATA);
+			sb_start_write(upper_sb);
 			if (data_pos > old_pos) {
 				hole_len = data_pos - old_pos;
 				len -= hole_len;
Signature · 1 line
2.34.1
author maintainer
#3 lore raw
In reply to Amir Goldstein
Quoted 14 lines · Amir Goldstein

Hi Miklos,

These prep patches are needed for my start-write-safe series [1]. This is not urgent and the prep patches don't need to be merged for next cycle, but I think these are good changes regardless, so wanted to post them for early review - if you like them you can queue them for 6.6.

It is quite hard to do the review of the locking reorder patch from the diff itself and I couldn't figure out a better way to split this change. I've intentionally left some otherwise useless out: goto labels to make the patch review a bit simper - they could be removed later.

Quoted 4 lines

On the good side, lockdep was very tough with me and it easily detected bugs in the earlier versions of the patches.

Going on vacation. will be back round rc6.

Hi Miklos,

Have you had a chance to look at these changes?
I can queue them up in overlayfs-next if they are acceptable.

Thanks,
Amir.

Quoted 17 lines

[1] https://github.com/amir73il/linux/commits/start-write-safe

Amir Goldstein (2):
ovl: reorder ovl_want_write() after ovl_inode_lock()
ovl: avoid lockdep warning with open and llseek of lower file

 fs/overlayfs/copy_up.c | 34 +++++++++++++-------
 fs/overlayfs/dir.c     | 71 ++++++++++++++++++------------------------
 fs/overlayfs/export.c  |  7 +----
 fs/overlayfs/inode.c   | 56 ++++++++++++++++-----------------
 fs/overlayfs/util.c    | 18 +++++++++--
 5 files changed, 97 insertions(+), 89 deletions(-)

--
2.34.1

maintainer
#4 lore raw
In reply to Amir Goldstein
Quoted 19 lines · Amir Goldstein

Hi Miklos,

These prep patches are needed for my start-write-safe series [1]. This is not urgent and the prep patches don't need to be merged for next cycle, but I think these are good changes regardless, so wanted to post them for early review - if you like them you can queue them for 6.6.

It is quite hard to do the review of the locking reorder patch from the diff itself and I couldn't figure out a better way to split this change. I've intentionally left some otherwise useless out: goto labels to make the patch review a bit simper - they could be removed later.

On the good side, lockdep was very tough with me and it easily detected bugs in the earlier versions of the patches.

Going on vacation. will be back round rc6.

Quoted 7 lines

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/start-write-safe

Amir Goldstein (2):
ovl: reorder ovl_want_write() after ovl_inode_lock()

This one generally looks good. The failure paths will need careful review, because those are usually not exercised by the test suites.

Quoted 1 line

ovl: avoid lockdep warning with open and llseek of lower file

But I dislike this one. Seems like a bad workaround for a possibly non-issue. I understand the desire to silence lockdep, but surely we can do better.

Thanks,
Miklos

author maintainer
#5 lore raw
Quoted 29 lines · Miklos Szeredi
Quoted 19 lines · Amir Goldstein

Hi Miklos,

These prep patches are needed for my start-write-safe series [1]. This is not urgent and the prep patches don't need to be merged for next cycle, but I think these are good changes regardless, so wanted to post them for early review - if you like them you can queue them for 6.6.

It is quite hard to do the review of the locking reorder patch from the diff itself and I couldn't figure out a better way to split this change. I've intentionally left some otherwise useless out: goto labels to make the patch review a bit simper - they could be removed later.

On the good side, lockdep was very tough with me and it easily detected bugs in the earlier versions of the patches.

Going on vacation. will be back round rc6.

Quoted 7 lines

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/start-write-safe

Amir Goldstein (2):
ovl: reorder ovl_want_write() after ovl_inode_lock()

Quoted 8 lines

This one generally looks good. The failure paths will need careful review, because those are usually not exercised by the test suites.

Quoted 1 line

ovl: avoid lockdep warning with open and llseek of lower file

But I dislike this one. Seems like a bad workaround for a possibly non-issue. I understand the desire to silence lockdep, but surely we can do better.

Well, I didn't do it to silence lockdep. I did it as a prereq for start-write-safe fsnotify hooks (see [1] above). Silencing lockdep is an added bonus that I observed along the way.

v2 [2] has a less hacky, but more noisy version of this patch which minimizes the scope of ovl_want_write() to when we need it. Let me know if this is what you had in mind.

After looking more closely, I found another possible deadlock with nested overlay, which I think is real and added another commit to fix it.

[2] https://github.com/amir73il/linux/commits/ovl_want_write-v2

Thanks,
Amir.

maintainer
#6 lore raw
Quoted 7 lines · Amir Goldstein

Well, I didn't do it to silence lockdep. I did it as a prereq for start-write-safe fsnotify hooks (see [1] above). Silencing lockdep is an added bonus that I observed along the way.

v2 [2] has a less hacky, but more noisy version of this patch which minimizes the scope of ovl_want_write() to when we need it. Let me know if this is what you had in mind.

Yes, I like this better, thanks.

Please post this as a series, and I'll do a review. I've already
caught a glance of an error case bug in one of the patches.

Thanks,
Miklos