xfs: reserve quota for directory expansion when hardlinking files
PATCH 1/1 xfs: reserve quota for directory expansion when hardlinking files
From: Darrick J. Wong <[email protected]>
The XFS implementation of the linkat call does not reserve quota for the potential directory expansion. This means that we don't reject the expansion with EDQUOT when we're at or near a hard limit, which means that one can use linkat() to exceed quota. Fix this by adding a quota reservation.
--- fs/xfs/xfs_inode.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/xfs/xfs_inode.c +4 -0
@@ -1249,6 +1249,10 @@ xfs_link(xfs_trans_ijoin (tp, sip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, tdp,XFS_ILOCK_EXCL ); + error =xfs_trans_reserve_quota_nblks (tp, tdp, resblks,0 , false); +if (error) + goto error_return; + error =xfs_iext_count_may_overflow (tdp,XFS_DATA_FORK ,XFS_IEXT_DIR_MANIP_CNT (mp));if (error)
PATCH 1/1 generic: test that linking into a directory fails with EDQUOT
From: Darrick J. Wong <[email protected]>
Add a regression test to make sure that unprivileged userspace linking into a directory fails with EDQUOT when the directory quota limits have been exceeded.
--- tests/generic/732 | 67 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/732.out | 3 ++ 2 files changed, 70 insertions(+) create mode 100755 tests/generic/732 create mode 100644 tests/generic/732.out
tests/generic/732 new file +67 -0
@@ -0,0 +1,67 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Oracle. All Rights Reserved. +# +# FS QA Test No. 732 +# +# Ensure that unprivileged userspace hits EDQUOT while linking files into a +# directory when the directory's quota limits have been exceeded. +# +# Regression test for commit: +# +# XXXXXXXXXXXX ("xfs: reserve quota for directory expansion when hardlinking files") +# +. ./common/preamble +_begin_fstest auto quick quota + +# Import common functions. +. ./common/filter +. ./common/quota + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_require_quota +_require_user +_require_scratch + +_scratch_mkfs > "$seqres.full" 2>&1 +_qmount_option usrquota +_qmount + +blocksize=$(_get_block_size $SCRATCH_MNT) +scratchdir=$SCRATCH_MNT/dir +scratchfile=$SCRATCH_MNT/file +mkdir $scratchdir +touch $scratchfile + +# Create a 2-block directory for our 1-block quota limit +total_size=$((blocksize * 2)) +dirents=$((total_size / 255)) + +for ((i = 0; i < dirents; i++)); do + name=$(printf "x%0254d" $i) + ln $scratchfile $scratchdir/$name +done + +# Set a low quota hardlimit for an unprivileged uid and chown the files to it +echo "set up quota" >> $seqres.full +setquota -u $qa_user 0 "$((blocksize / 1024))" 0 0 $SCRATCH_MNT +chown $qa_user $scratchdir $scratchfile +repquota -upn $SCRATCH_MNT >> $seqres.full + +# Fail at appending the directory as qa_user to ensure quota enforcement works +echo "fail quota" >> $seqres.full +for ((i = 0; i < dirents; i++)); do + name=$(printf "y%0254d" $i) + su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \ + _filter_scratch | sed -e 's/y[0-9]*/yXXX/g' + test "${PIPESTATUS[0]}" -ne 0 && break +done +repquota -upn $SCRATCH_MNT >> $seqres.full + +# success, all done +echo Silence is golden +status=0 +exit
tests/generic/732.out new file +3 -0
@@ -0,0 +1,3 @@ +QA output created by 732 +ln: failed to create hard link 'SCRATCH_MNT/dir/yXXX': Disk quota exceeded +Silence is golden
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The XFS implementation of the linkat call does not reserve quota for the potential directory expansion. This means that we don't reject the expansion with EDQUOT when we're at or near a hard limit, which means that one can use
linkat()to exceed quota. Fix this by adding a quota reservation.
Could someone review this, please? I'd like to get a 5.18-fixes branch rolling along with the other patches that have already been sent.
--D
Quoted 20 lines
Signed-off-by Darrick J. Wong <[email protected]>--- fs/xfs/xfs_inode.c | 4 ++++ 1 file changed, 4 insertions(+)fs/xfs/xfs_inode.c +4
-0@@ -1249,6 +1249,10 @@ xfs_link(xfs_trans_ijoin (tp, sip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, tdp,XFS_ILOCK_EXCL ); + error =xfs_trans_reserve_quota_nblks (tp, tdp, resblks,0 , false); +if (error) + goto error_return; + error =xfs_iext_count_may_overflow (tdp,XFS_DATA_FORK ,XFS_IEXT_DIR_MANIP_CNT (mp));if (error)
Quoted 18 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The XFS implementation of the linkat call does not reserve quota for the potential directory expansion. This means that we don't reject the expansion with EDQUOT when we're at or near a hard limit, which means that one can use
linkat()to exceed quota. Fix this by adding a quota reservation.Signed-off-by Darrick J. Wong <[email protected]>--- fs/xfs/xfs_inode.c | 4 ++++ 1 file changed, 4 insertions(+)fs/xfs/xfs_inode.c +0
-0
fs/xfs/xfs_inode.c xfs_link( · line 1258
Quoted 10 lines
fs/xfs/xfs_inode.c +4
-0@@ -1249,6 +1249,10 @@ xfs_link(xfs_trans_ijoin (tp, sip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, tdp,XFS_ILOCK_EXCL ); + error =xfs_trans_reserve_quota_nblks (tp, tdp, resblks,0 , false); +if (error) + goto error_return; + error =xfs_iext_count_may_overflow (tdp,XFS_DATA_FORK ,XFS_IEXT_DIR_MANIP_CNT (mp));if (error)
Yup, ok, but doesn't xfs_remove have exactly the same problem? i.e. removing a directory entry can punch a hole in the bmbt and require new allocations for a BMBT split, thereby increasing the number of blocks allocated to the directory? e.g. remove a single data block, need to then allocate half a dozen BMBT blocks for the shape change.
If so, then both xfs_link() and xfs_remove() have exactly the same dquot, inode locking and transaction setup code and requirements, and probably should be factored out into xfs_trans_alloc_dir() (i.e. equivalent of xfs_trans_alloc_icreate() used by all the inode create functions). That way we only have one copy of this preamble and only need to fix the bug in one place?
Alternatively, fix the bug in both places first and add a followup patch that factors out this code as per above.
Hmmm - looking further a callers of xfs_lock_two_inodes(), it would appear that xfs_swap_extents() needs the same quota reservation and also largely has the same transaction setup and inode locking preamble as link and remove...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 22 lines · Dave Chinner
Quoted 21 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The XFS implementation of the linkat call does not reserve quota for the potential directory expansion. This means that we don't reject the expansion with EDQUOT when we're at or near a hard limit, which means that one can use
linkat()to exceed quota. Fix this by adding a quota reservation.Signed-off-by Darrick J. Wong <[email protected]>--- fs/xfs/xfs_inode.c | 4 ++++ 1 file changed, 4 insertions(+)fs/xfs/xfs_inode.c +0
-0@@ -1249,6 +1249,10 @@ xfs_link(xfs_trans_ijoin (tp, sip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, tdp,XFS_ILOCK_EXCL );
Quoted 9 lines
fs/xfs/xfs_inode.c xfs_link( · line 1258
Quoted 7 lines
fs/xfs/xfs_inode.c +4
-0+ error =xfs_trans_reserve_quota_nblks (tp, tdp, resblks,0 , false); +if (error) + goto error_return; + error =xfs_iext_count_may_overflow (tdp,XFS_DATA_FORK ,XFS_IEXT_DIR_MANIP_CNT (mp));if (error)Yup, ok, but doesn't
xfs_removehave exactly the same problem? i.e.
Yes, it does, however, the reason I don't have a fix for that ready is that...
Quoted 2 lines
removing a directory entry can punch a hole in the bmbt and require new allocations for a BMBT split, thereby increasing the number of
...rejecting a directory unlink with EDQUOT creates the situation where a user who's gone over the soft limit cannot rm a file to get themselves back under quota because the removal asked for enough bmbt-expansion quota reservation to push the quota over the hard limit...
Quoted 2 lines
blocks allocated to the directory? e.g. remove a single data block, need to then allocate half a dozen BMBT blocks for the shape change.
...and while the next thing that occurred to me was to retry the quota reservation with FORCE_RES, having such a path means that one can still overrun the hard limit (albeit slowly) by creating a fragmented directory and selectively removing entries to cause bmbt splits.
I /think/ I'm ok with the "retry with FORCE_QUOTA" solution for xfs_remove, but I'm hanging onto it for now for further consideration and QA testing.
Quoted 6 lines
If so, then both
xfs_link()andxfs_remove()have exactly the same dquot, inode locking and transaction setup code and requirements, and probably should be factored out intoxfs_trans_alloc_dir()(i.e. equivalent ofxfs_trans_alloc_icreate()used by all the inode create functions). That way we only have one copy of this preamble and only need to fix the bug in one place?
They're not the same problem -- adding hardlinks is not a known strategy for reducing quota usage below the limits, whereas unlinking files is.
Quoted 2 lines
Alternatively, fix the bug in both places first and add a followup patch that factors out this code as per above.
I sent a patch for the link situation because I thought it looked like an obvious fix, and left the unlink() problem until a full solution is presented or proved impossible.
Quoted 4 lines
Hmmm - looking further a callers of
xfs_lock_two_inodes(), it would appear thatxfs_swap_extents()needs the same quota reservation and also largely has the same transaction setup and inode locking preamble as link and remove...
Yes, I know about that problem. I've *solved* that problem with the atomic extent swap rewrite that's been hanging out in djwong-dev since late 2019 as part of the online fsck series. Perhaps I will have time to send that in late 2022.
--D
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 23 lines · Darrick J. Wong
Quoted 22 lines · Dave Chinner
Quoted 21 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The XFS implementation of the linkat call does not reserve quota for the potential directory expansion. This means that we don't reject the expansion with EDQUOT when we're at or near a hard limit, which means that one can use
linkat()to exceed quota. Fix this by adding a quota reservation.Signed-off-by Darrick J. Wong <[email protected]>--- fs/xfs/xfs_inode.c | 4 ++++ 1 file changed, 4 insertions(+)fs/xfs/xfs_inode.c +0
-0@@ -1249,6 +1249,10 @@ xfs_link(xfs_trans_ijoin (tp, sip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, tdp,XFS_ILOCK_EXCL );
Quoted 4 lines
Quoted 4 lines
fs/xfs/xfs_inode.c xfs_link( · line 1255
Quoted 4 lines
fs/xfs/xfs_inode.c +4
-0+ error =xfs_trans_reserve_quota_nblks (tp, tdp, resblks,0 , false); +if (error) + goto error_return; +
Hmmm - I just noticed that trans_alloc_icreate and trans_alloc_inode also run a blockgc pass on EDQUOT or ENOSPC when they fail to reserve quota to try to free up some space before retrying. Do we need that here, too?
Quoted 9 lines
Quoted 5 lines
fs/xfs/xfs_inode.c xfs_link( · line 1258
Quoted 3 lines
fs/xfs/xfs_inode.c +0
-0error =xfs_iext_count_may_overflow (tdp,XFS_DATA_FORK ,XFS_IEXT_DIR_MANIP_CNT (mp));if (error)Yup, ok, but doesn't
xfs_removehave exactly the same problem? i.e.Yes, it does, however, the reason I don't have a fix for that ready is that...
Quoted 7 lines
Quoted 2 lines
removing a directory entry can punch a hole in the bmbt and require new allocations for a BMBT split, thereby increasing the number of
...rejecting a directory unlink with EDQUOT creates the situation where a user who's gone over the soft limit cannot rm a file to get themselves back under quota because the removal asked for enough bmbt-expansion quota reservation to push the quota over the hard limit...
Both link and remove already have "zero reservation" paths for ENOSPC - if they are to be made quota aware they'll end up with resblks = 0 and so xfs_trans_reserve_quota_nblks() is a no-op at ENOSPC. So ....
Quoted 4 lines
Quoted 3 lines
blocks allocated to the directory? e.g. remove a single data block, need to then allocate half a dozen BMBT blocks for the shape change.
Quoted 8 lines
...and while the next thing that occurred to me was to retry the quota reservation with
FORCE_RES, having such a path means that one can still overrun the hard limit (albeit slowly) by creating a fragmented directory and selectively removing entries to cause bmbt splits.I /think/ I'm ok with the "retry with
FORCE_QUOTA" solution forxfs_remove, but I'm hanging onto it for now for further consideration and QA testing.
... yes, I think this would be just fine. I don't think we really care in any way about people trying to grow their quota beyond the hard limit by a few blocks by intentionally fragmenting really large directories. If their quota allows them directories and inode counts large enough for this to be an avenue to exceeding hard quota limits by a block or two, nobody is going to notice about a block or two or extra space usage.
Quoted 10 lines
Quoted 6 lines
If so, then both
xfs_link()andxfs_remove()have exactly the same dquot, inode locking and transaction setup code and requirements, and probably should be factored out intoxfs_trans_alloc_dir()(i.e. equivalent ofxfs_trans_alloc_icreate()used by all the inode create functions). That way we only have one copy of this preamble and only need to fix the bug in one place?They're not the same problem -- adding hardlinks is not a known strategy for reducing quota usage below the limits, whereas unlinking files is.
Quoted 6 lines
Quoted 2 lines
Alternatively, fix the bug in both places first and add a followup patch that factors out this code as per above.
I sent a patch for the link situation because I thought it looked like an obvious fix, and left the
unlink()problem until a full solution is presented or proved impossible.
Ok. None of this was mentioned in the patch, so I had no idea about any of the things you are doing behind the scenes. I simply saw the same problem in other places....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 24 lines · Dave Chinner
Quoted 23 lines · Darrick J. Wong
Quoted 22 lines · Dave Chinner
Quoted 21 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The XFS implementation of the linkat call does not reserve quota for the potential directory expansion. This means that we don't reject the expansion with EDQUOT when we're at or near a hard limit, which means that one can use
linkat()to exceed quota. Fix this by adding a quota reservation.Signed-off-by Darrick J. Wong <[email protected]>--- fs/xfs/xfs_inode.c | 4 ++++ 1 file changed, 4 insertions(+)fs/xfs/xfs_inode.c +0
-0@@ -1249,6 +1249,10 @@ xfs_link(xfs_trans_ijoin (tp, sip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, tdp,XFS_ILOCK_EXCL );
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
fs/xfs/xfs_inode.c xfs_link( · line 1255
Quoted 4 lines
fs/xfs/xfs_inode.c +4
-0+ error =xfs_trans_reserve_quota_nblks (tp, tdp, resblks,0 , false); +if (error) + goto error_return; +Hmmm - I just noticed that
trans_alloc_icreateandtrans_alloc_inodealso run a blockgc pass on EDQUOT or ENOSPC when they fail to reserve quota to try to free up some space before retrying. Do we need that here, too?
(Re)trying to clear more space sounds like a good idea.
Quoted 35 lines
Quoted 9 lines
Quoted 5 lines
fs/xfs/xfs_inode.c xfs_link( · line 1258
Quoted 3 lines
fs/xfs/xfs_inode.c +0
-0error =xfs_iext_count_may_overflow (tdp,XFS_DATA_FORK ,XFS_IEXT_DIR_MANIP_CNT (mp));if (error)Yup, ok, but doesn't
xfs_removehave exactly the same problem? i.e.Yes, it does, however, the reason I don't have a fix for that ready is that...
Quoted 7 lines
Quoted 2 lines
removing a directory entry can punch a hole in the bmbt and require new allocations for a BMBT split, thereby increasing the number of
...rejecting a directory unlink with EDQUOT creates the situation where a user who's gone over the soft limit cannot rm a file to get themselves back under quota because the removal asked for enough bmbt-expansion quota reservation to push the quota over the hard limit...
Both link and remove already have "zero reservation" paths for ENOSPC - if they are to be made quota aware they'll end up with resblks = 0 and so
xfs_trans_reserve_quota_nblks()is a no-op at ENOSPC. So ....Quoted 4 lines
Quoted 3 lines
blocks allocated to the directory? e.g. remove a single data block, need to then allocate half a dozen BMBT blocks for the shape change.
Quoted 8 lines
...and while the next thing that occurred to me was to retry the quota reservation with
FORCE_RES, having such a path means that one can still overrun the hard limit (albeit slowly) by creating a fragmented directory and selectively removing entries to cause bmbt splits.I /think/ I'm ok with the "retry with
FORCE_QUOTA" solution forxfs_remove, but I'm hanging onto it for now for further consideration and QA testing.
Quoted 7 lines
... yes, I think this would be just fine. I don't think we really care in any way about people trying to grow their quota beyond the hard limit by a few blocks by intentionally fragmenting really large directories. If their quota allows them directories and inode counts large enough for this to be an avenue to exceeding hard quota limits by a block or two, nobody is going to notice about a block or two or extra space usage.
At least for the link case, you can trivially continue to expand the directory by hardlinking the same file over and over. Part of the weirdness here might be related to the fact that a transaction with no quota reservation is allowed to commit the quota usage changes, even if that would bump them past the limit.
Hm. Perhaps the trick here should be that we reduce resblks to zero for ENOSPC or EDQUOT, which means that you can continue link()ing files into a directory so long as it won't cause the dir to expand. xfs_remove (aka unlink()) handles reservationless removals by deferring the directory shrink operation if there isn't space, so I think it can be ported to use the new "alloc and reserve" function too.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
If so, then both
xfs_link()andxfs_remove()have exactly the same dquot, inode locking and transaction setup code and requirements, and probably should be factored out intoxfs_trans_alloc_dir()(i.e. equivalent ofxfs_trans_alloc_icreate()used by all the inode create functions). That way we only have one copy of this preamble and only need to fix the bug in one place?They're not the same problem -- adding hardlinks is not a known strategy for reducing quota usage below the limits, whereas unlinking files is.
Quoted 2 lines
Alternatively, fix the bug in both places first and add a followup patch that factors out this code as per above.
Quoted 7 lines
Quoted 3 lines
I sent a patch for the link situation because I thought it looked like an obvious fix, and left the
unlink()problem until a full solution is presented or proved impossible.Ok. None of this was mentioned in the patch, so I had no idea about any of the things you are doing behind the scenes. I simply saw the same problem in other places....
Yeah, there are more fiddly fixes for setattr coming down the line too...
--D
Quoted 7 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]