Skip to contentSkip to conversation

buffered write path without inode lock (for bcachefs)

PATCH In review Kent Overstreet 3 participants 9 messages Feb 29, 2024 VFS

lore
Share & tools
lore
#0 lore raw

cover buffered write path without inode lock (for bcachefs)

this is going in my for-next branch - it's tested and I think all the corner cases are handled to my satisfaction (there are some fun ones!)

Kent Overstreet (2):
fs: file_remove_privs_flags()
bcachefs: Buffered write path now can avoid the inode lock

 fs/bcachefs/errcode.h        |   3 +-
 fs/bcachefs/fs-io-buffered.c | 145 +++++++++++++++++++++++++----------
 fs/inode.c                   |   7 +-
 include/linux/fs.h           |   1 +
 4 files changed, 112 insertions(+), 44 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/2 fs: file_remove_privs_flags()

Rename and export __file_remove_privs(); for a buffered write path that doesn't take the inode lock we need to be able to check if the operation needs to do work first.

Cc Alexander Viro <[email protected]>
Cc Christian Brauner <[email protected]>
---
 fs/inode.c         | 7 ++++---
 include/linux/fs.h | 1 +
 2 files changed, 5 insertions(+), 3 deletions(-)
fs/inode.c +4 -3
index 91048c4c9c9e..b465afdbfcef 100644
@@ -2031,7 +2031,7 @@ static int __remove_privs(struct mnt_idmap *idmap,
 	return notify_change(idmap, dentry, &newattrs, NULL);
 }
 
-static int __file_remove_privs(struct file *file, unsigned int flags)
+int file_remove_privs_flags(struct file *file, unsigned int flags)
 {
 	struct dentry *dentry = file_dentry(file);
 	struct inode *inode = file_inode(file);
@@ -2056,6 +2056,7 @@ static int __file_remove_privs(struct file *file, unsigned int flags)
 		inode_has_no_xattr(inode);
 	return error;
 }
+EXPORT_SYMBOL_GPL(file_remove_privs_flags);
 
 /**
  * file_remove_privs - remove special file privileges (suid, capabilities)
@@ -2068,7 +2069,7 @@ static int __file_remove_privs(struct file *file, unsigned int flags)
  */
 int file_remove_privs(struct file *file)
 {
-	return __file_remove_privs(file, 0);
+	return file_remove_privs_flags(file, 0);
 }
 EXPORT_SYMBOL(file_remove_privs);
 
@@ -2161,7 +2162,7 @@ static int file_modified_flags(struct file *file, int flags)
 	 * Clear the security bits if the process is not being run by root.
 	 * This keeps people from modifying setuid and setgid binaries.
 	 */
-	ret = __file_remove_privs(file, flags);
+	ret = file_remove_privs_flags(file, flags);
 	if (ret)
 		return ret;
 
include/linux/fs.h +1 -0
index 1fbc72c5f112..14ea66b62823 100644
@@ -3004,6 +3004,7 @@ extern struct inode *new_inode_pseudo(struct super_block *sb);
 extern struct inode *new_inode(struct super_block *sb);
 extern void free_inode_nonrcu(struct inode *inode);
 extern int setattr_should_drop_suidgid(struct mnt_idmap *, struct inode *);
+extern int file_remove_privs_flags(struct file *file, unsigned int flags);
 extern int file_remove_privs(struct file *);
 int setattr_should_drop_sgid(struct mnt_idmap *idmap,
 			     const struct inode *inode);
Signature · 1 line
2.43.0
#2 lore raw

PATCH 2/2 bcachefs: Buffered write path now can avoid the inode lock

In reply to Kent Overstreet

Non append, non extending buffered writes can now avoid taking the inode lock.

To ensure atomicity of writes w.r.t. other writes, we lock every folio that we'll be writing to, and if this fails we fall back to taking the inode lock.

Extensive comments are provided as to corner cases.

---
 fs/bcachefs/errcode.h        |   3 +-
 fs/bcachefs/fs-io-buffered.c | 145 +++++++++++++++++++++++++----------
 2 files changed, 107 insertions(+), 41 deletions(-)
fs/bcachefs/errcode.h +2 -1
index e960a6eae66a..2791d5127090 100644
@@ -247,7 +247,8 @@
 	x(BCH_ERR_nopromote,		nopromote_congested)			\
 	x(BCH_ERR_nopromote,		nopromote_in_flight)			\
 	x(BCH_ERR_nopromote,		nopromote_no_writes)			\
-	x(BCH_ERR_nopromote,		nopromote_enomem)
+	x(BCH_ERR_nopromote,		nopromote_enomem)			\
+	x(0,				need_inode_lock)
 
 enum bch_errcode {
 	BCH_ERR_START		= 2048,
fs/bcachefs/fs-io-buffered.c +105 -40

197 lines not on this page · raw message

Signature · 1 line
2.43.0
#3 lore raw

Re: [PATCH 2/2] bcachefs: Buffered write path now can avoid the inode lock

Quoted 3 lines · Kent Overstreet

Non append, non extending buffered writes can now avoid taking the inode lock.

I think this is buggy.

I think you still need to take the inode lock *shared* for the writes, because otherwise you can have somebody else that truncates the file and now you will do a write past the end of the size of the file. That will cause a lot of issues.

So it's not a "inode_lock or not" situation. I think it's a "inode_lock vs inode_locks_shared" situation.

Note that the reading side isn't all that critical - if a read races with a truncate, at worst it will read some zeroes because we used the old length and the page cache got cleared in the meantime.

But the writing side ends up having actual consistency issues on disk. You don't want to have a truncate that removes the pages past the end of the new size and clears the end of the new last page, and race with another write that used the old size and *thought* it was writing to the middle of the file, but is now actually accessing a folio that is past the end of the whole file and writing to it.

There may be some reason that I'm missing that would make this a non-issue, but I really think you want to get the inode lock at least shared for the duration of the write.

Also note that for similar reasons, you can't just look at "will I extend the file" and take the lock non-shared. No, in order to actually trust the size, you need to *hold* the lock, so the logic needs to be something like

- take the lock exclusively if O_APPEND or if it *looks* like you
might extend the file size.

- otherwise, take the shared lock, and THEN RE-CHECK. The file size
might have changed, so now you need to double-check that you're really not going to extend the size of the file, and if you are, you need to go back and take the inode lock exclusively after all.

Again - I haven't thought a ton about this, so maybe there's some trick to it, but the above is what my naive thinking says the rule has to be. Writes are different from reads.

Linus

#4 lore raw

Re: [PATCH 2/2] bcachefs: Buffered write path now can avoid the inode lock

Quoted 9 lines · Linus Torvalds

- take the lock exclusively if O_APPEND or if it *looks* like you
might extend the file size.

- otherwise, take the shared lock, and THEN RE-CHECK. The file size
might have changed, so now you need to double-check that you're really not going to extend the size of the file, and if you are, you need to go back and take the inode lock exclusively after all.

Same goes for the suid/sgid checks. You need to take the inode lock shared to even have the i_mode be stable, and at that point you might decide "oh, I need to clear suid/sgid after all" and have to go drop the lock and retake it for exclusive access after all.

(And yes, we *really* should have a rwsem_try_upgrade() operation and at least avoid the "read_unlock -> write_lock" dance), but we don't. See a comment in mmap_upgrade_trylock() about it. It should be reasonably easy to add, but we've never really had enough reason to. Maybe somebody decides it's worth the effort)

Linus

#5 lore raw

Re: [PATCH 2/2] bcachefs: Buffered write path now can avoid the inode lock

Quoted 12 lines · Linus Torvalds
Quoted 9 lines · Linus Torvalds

- take the lock exclusively if O_APPEND or if it *looks* like you
might extend the file size.

- otherwise, take the shared lock, and THEN RE-CHECK. The file size
might have changed, so now you need to double-check that you're really not going to extend the size of the file, and if you are, you need to go back and take the inode lock exclusively after all.

Quoted 4 lines

Same goes for the suid/sgid checks. You need to take the inode lock shared to even have the i_mode be stable, and at that point you might decide "oh, I need to clear suid/sgid after all" and have to go drop the lock and retake it for exclusive access after all.

Do we though? Yes i_mode can change mid write, but if userspace issues a chmod() while we're in a write() - the end result has to be consistent with either "chmod(); write();" or 'write(); chmod();"; meaning as long as there's been a barrier in the syscall path so that we can't have seen a value of i_mode from before the chmod returned we're good.

#6 lore raw

Re: [PATCH 2/2] bcachefs: Buffered write path now can avoid the inode lock

In reply to Linus Torvalds
Quoted 12 lines · Linus Torvalds
Quoted 9 lines · Linus Torvalds

- take the lock exclusively if O_APPEND or if it *looks* like you
might extend the file size.

- otherwise, take the shared lock, and THEN RE-CHECK. The file size
might have changed, so now you need to double-check that you're really not going to extend the size of the file, and if you are, you need to go back and take the inode lock exclusively after all.

Quoted 4 lines

Same goes for the suid/sgid checks. You need to take the inode lock shared to even have the i_mode be stable, and at that point you might decide "oh, I need to clear suid/sgid after all" and have to go drop the lock and retake it for exclusive access after all.

I agree. And this is how we've done it in xfs as well. The inode lock is taken shared, then check for IS_NOSEC(inode) and if that's true keep the shared lock, otherwise upgrade to an exclusive lock. Note that this whole check also checks filesystem capability xattrs.

I think you'd also get potential security issues or at least very subtle behavior. Someone could make a binary setuid and a concurrent write happens. chmod is taking the exclusive lock and there's a concurrent write going on changing the contents to something unsafe without being forced to remove the set*id bits.

Similar for {g,u}id changes. Someone starts a chown() taking inode lock while someone issues a concurrent write. The chown changes the group ownership so that a writer would be forced to drop the sgid bit. But the writer proceeds keeping that bit.

So that's all potentially pretty subtle side-effects we'd allow. And I've removed so many bugs in that area the last couple of years as well. So let's please use the shared lock.

#7 lore raw

Re: [PATCH 2/2] bcachefs: Buffered write path now can avoid the inode lock

Quoted 29 lines · Christian Brauner
Quoted 12 lines · Linus Torvalds
Quoted 9 lines · Linus Torvalds

- take the lock exclusively if O_APPEND or if it *looks* like you
might extend the file size.

- otherwise, take the shared lock, and THEN RE-CHECK. The file size
might have changed, so now you need to double-check that you're really not going to extend the size of the file, and if you are, you need to go back and take the inode lock exclusively after all.

Quoted 4 lines

Same goes for the suid/sgid checks. You need to take the inode lock shared to even have the i_mode be stable, and at that point you might decide "oh, I need to clear suid/sgid after all" and have to go drop the lock and retake it for exclusive access after all.

I agree. And this is how we've done it in xfs as well. The inode lock is taken shared, then check for IS_NOSEC(inode) and if that's true keep the shared lock, otherwise upgrade to an exclusive lock. Note that this whole check also checks filesystem capability xattrs.

I think you'd also get potential security issues or at least very subtle behavior. Someone could make a binary setuid and a concurrent write happens. chmod is taking the exclusive lock and there's a concurrent write going on changing the contents to something unsafe without being forced to remove the set*id bits.

Quoted 4 lines

Similar for {g,u}id changes. Someone starts a chown() taking inode lock while someone issues a concurrent write. The chown changes the group ownership so that a writer would be forced to drop the sgid bit. But the writer proceeds keeping that bit.

That's the kind of race that's not actually a race - if the syscalls being invoked at the same time, it's fine if the result is as if the write happened first, then the chown.

For there to be an actual race one of two things would have to be the case:
- the chown/chmod path would have to be dependent on the on the data the write path is changing, or
- there'd have to be a problem wnith the write path seeing inconsistent state mid modification

But I can't claim 100% the second case isn't a factor, I don't know the security side of things as well, and the XFS way sounds nice and clean.

#8 lore raw

Re: [PATCH 2/2] bcachefs: Buffered write path now can avoid the inode lock

In reply to Linus Torvalds
Quoted 26 lines · Linus Torvalds
Quoted 3 lines · Kent Overstreet

Non append, non extending buffered writes can now avoid taking the inode lock.

I think this is buggy.

I think you still need to take the inode lock *shared* for the writes, because otherwise you can have somebody else that truncates the file and now you will do a write past the end of the size of the file. That will cause a lot of issues.

So it's not a "inode_lock or not" situation. I think it's a "inode_lock vs inode_locks_shared" situation.

Note that the reading side isn't all that critical - if a read races with a truncate, at worst it will read some zeroes because we used the old length and the page cache got cleared in the meantime.

But the writing side ends up having actual consistency issues on disk. You don't want to have a truncate that removes the pages past the end of the new size and clears the end of the new last page, and race with another write that used the old size and *thought* it was writing to the middle of the file, but is now actually accessing a folio that is past the end of the whole file and writing to it.

Quoted 3 lines

There may be some reason that I'm missing that would make this a non-issue, but I really think you want to get the inode lock at least shared for the duration of the write.

It's even mentioned in one of the comments - bcachefs's pagecache add lock guards against that. The rules for that lock are

- things that add to the pagecache take the add side of that lock
- things that remove the pagecache take the block side of that lock

I added that so that we wouldn't have pagecache inconsistency issues with dio and mmap'd IO - without it anything that needs to shoot down the pagecache while it's doing IO that bypasses the pagecache is buggy (fpunch, fcollapse...).

Quoted 5 lines

Also note that for similar reasons, you can't just look at "will I extend the file" and take the lock non-shared. No, in order to actually trust the size, you need to *hold* the lock, so the logic needs to be something like

Quoted 7 lines

- take the lock exclusively if O_APPEND or if it *looks* like you
might extend the file size.

- otherwise, take the shared lock, and THEN RE-CHECK. The file size
might have changed, so now you need to double-check that you're really not going to extend the size of the file, and if you are, you need to go back and take the inode lock exclusively after all.

That one - yes.

pagecache add lock was also supposed to handle that because anything that changes i_size downward needs pagecache block, but I moved where we take that lock for lock ordering reasons, and I really didn't need too...

I'm undecided on that one. I dislike using pagecache add lock to guard i_size because that's really not what it's for, but I also hate hitting the inode lock if we don't actually need it.

Kinda waiting for Al to drop in and mention the other super obscure reason the inode lock actually is needed here...