Skip to contentSkip to conversation

lock i_mutex for fallocate?

DISCUSSION Allison Henderson 5 participants 7 messages Sep 1, 2011 VFS

lore
Share & tools
lore
#0 lore raw

Hi All,

In ext4 punch hole, we realized that the punch hole operation needs to be done under i_mutex just like truncate. i_mutex for truncate is held in the vfs layer, so we dont need to lock it at the file system layer, but vfs does not lock i_mutex for fallocate. We can lock i_mutex for fallocate at the fs layer, but question was raised then: should i_mutex for fallocate be held in the vfs layer instead? I do not know if other file systems need i_mutex to be locked for fallocate, or if they might be locking it already, so I am doing some investigating on this idea, and also the appropriate use of i_mutex in general. Can someone provide some insight this topic? Thx!

Allison Henderson

#1 lore raw

Oh, I meant for this to go to linux-fsdevel instead of linux-kernel, but all feedback is welcome! :)

Quoted 14 lines · Allison Henderson

Hi All,

In ext4 punch hole, we realized that the punch hole operation needs to be done under i_mutex just like truncate. i_mutex for truncate is held in the vfs layer, so we dont need to lock it at the file system layer, but vfs does not lock i_mutex for fallocate. We can lock i_mutex for fallocate at the fs layer, but question was raised then: should i_mutex for fallocate be held in the vfs layer instead? I do not know if other file systems need i_mutex to be locked for fallocate, or if they might be locking it already, so I am doing some investigating on this idea, and also the appropriate use of i_mutex in general. Can someone provide some insight this topic? Thx!

Allison Henderson

#2 lore raw
Quoted 17 lines · Allison Henderson

Oh, I meant for this to go to linux-fsdevel instead of linux-kernel, but all feedback is welcome! :)

Quoted 13 lines · Allison Henderson

Hi All,

In ext4 punch hole, we realized that the punch hole operation needs to be done under i_mutex just like truncate. i_mutex for truncate is held in the vfs layer, so we dont need to lock it at the file system layer, but vfs does not lock i_mutex for fallocate. We can lock i_mutex for fallocate at the fs layer, but question was raised then: should i_mutex for fallocate be held in the vfs layer instead? I do not know if other file systems need i_mutex to be locked for fallocate, or if they might be locking it already, so I am doing some investigating on this idea, and also the appropriate use of i_mutex in general. Can someone provide some insight this topic? Thx!

Btrfs has range locking for our extents, so we don't really need to be holding the i_mutex, even tho it appears we do it anyway. So I'd rather this not be moved to VFS for us who can do more fine grained locking. Thanks,

Josef

#3 lore raw
In reply to Allison Henderson

Hi Allison,

there are races between read and punch hole too. After pages are released and before punch hole hold i_data_sem, a page can be mapped by read, then write can be done on the page.

Maybe we should separate punch hole from fallocate, and solve the races in vfs layer.

Yongqiang.

Quoted 19 lines · Allison Henderson

Hi All,

In ext4 punch hole, we realized that the punch hole operation needs to be done under i_mutex just like truncate.  i_mutex for truncate is held in the vfs layer, so we dont need to lock it at the file system layer, but vfs does not lock i_mutex for fallocate.  We can lock i_mutex for fallocate at the fs layer, but question was raised then: should i_mutex for fallocate be held in the vfs layer instead?  I do not know if other file systems need i_mutex to be locked for fallocate, or if they might be locking it already, so I am doing some investigating on this idea, and also the appropriate use of i_mutex in general.  Can someone provide some insight this topic?  Thx!

Allison Henderson
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Signature · 6 lines
Best Wishes
Yongqiang Yang
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#4 lore raw
In reply to Allison Henderson
Quoted 8 lines · Allison Henderson

Hi All,

In ext4 punch hole, we realized that the punch hole operation needs to be done under i_mutex just like truncate. i_mutex for truncate is held in the vfs layer, so we dont need to lock it at the file system layer, but vfs does not lock i_mutex for fallocate. We can lock i_mutex for fallocate at the fs layer, but question was raised
then: should i_mutex for fallocate be held in the vfs layer instead?

No.

Quoted 2 lines

I do not know if other file systems need i_mutex to be locked for fallocate,

For one, XFS does not require i_mutex to be held for any extent manipulation of any kind (allocation, truncation, hole punch, unwritten extent conversion, etc).

Hence the current structure of having the filesystem take i_mutex if it needs it to protect allocations against races is appropriate.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#5 lore raw
In reply to Allison Henderson
Quoted 12 lines · Allison Henderson

Hi All,

In ext4 punch hole, we realized that the punch hole operation needs to be done under i_mutex just like truncate. i_mutex for truncate is held in the vfs layer, so we dont need to lock it at the file system layer, but vfs does not lock i_mutex for fallocate. We can lock i_mutex for fallocate at the fs layer, but question was raised
then: should i_mutex for fallocate be held in the vfs layer instead? I do not know if other file systems need i_mutex to be locked for fallocate, or if they might be locking it already, so I am doing some investigating on this idea, and also the appropriate use of i_mutex in general. Can someone provide some insight this topic?

Don't do it.

i_mutex is already overloaded, and this does not fit into any of the somewhat reasonable uses cases for it, which are:

a) for directories the VFS uses it to protect the tree topology
b) for regular files all generic I/O code currently uses it to serialize writers.
c) the VFS uses it around truncate, and setxattr updates
d) filesystems abuse it for internal metadata in various places

As you can see right now we do not hold it over any file operation, and I'm absolutely against adding that. I'd rather untange the current uses, specificly:

- push synchronization of setattr into the filesystems
- push synchronization of xattr write operations into the filesystems
- move the read/write synchronization to a separate shared/exclusive lock like it's already done in XFS, and like Lukas proposed for ext4. This fixes the Posix compliance corner cases about reads beeing atomic vs writes, simplifies direct I/O locking a lot, and allows for more parallel direct I/O support like XFS supports.
- try to get rid of the abuses inside filesystems as much as possible.

#6 lore raw
Quoted 29 lines · Christoph Hellwig
Quoted 12 lines · Allison Henderson

Hi All,

In ext4 punch hole, we realized that the punch hole operation needs to be done under i_mutex just like truncate. i_mutex for truncate is held in the vfs layer, so we dont need to lock it at the file system layer, but vfs does not lock i_mutex for fallocate. We can lock i_mutex for fallocate at the fs layer, but question was raised
then: should i_mutex for fallocate be held in the vfs layer instead? I do not know if other file systems need i_mutex to be locked for fallocate, or if they might be locking it already, so I am doing some investigating on this idea, and also the appropriate use of i_mutex in general. Can someone provide some insight this topic?

Don't do it.

i_mutex is already overloaded, and this does not fit into any of the somewhat reasonable uses cases for it, which are:

a) for directories the VFS uses it to protect the tree topology
b) for regular files all generic I/O code currently uses it to serialize writers.
c) the VFS uses it around truncate, and setxattr updates
d) filesystems abuse it for internal metadata in various places

As you can see right now we do not hold it over any file operation, and I'm absolutely against adding that. I'd rather untange the current uses, specificly:

Quoted 9 lines

- push synchronization of setattr into the filesystems
- push synchronization of xattr write operations into the filesystems
- move the read/write synchronization to a separate shared/exclusive lock like it's already done in XFS, and like Lukas proposed for ext4. This fixes the Posix compliance corner cases about reads beeing atomic vs writes, simplifies direct I/O locking a lot, and allows for more parallel direct I/O support like XFS supports.
- try to get rid of the abuses inside filesystems as much as possible.

Alrighty, this helps explain things! Thx all for the feedback! :)