Parent Pointers
v2 cover Parent Pointers
Hi all,
This is a rebase and resend of the latest parent pointer attributes for xfs. The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, or any other feature that could make use of quickly deriving an inodes path from the mount point.
This set can be viewed on github here
https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv2_rebase
And the corresponding xfsprogs code is here
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrsv2
This set has been tested with the below parent pointers tests
https://www.spinics.net/lists/fstests/msg19963.html
Updates since v1:
xfs: Fix multi-transaction larp replay
Resend (from stand alone patch)
xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
Increased XFS_DEFER_OPS_NR_INODES from 4 to 5 Moved to beginning of the set
Added code in xfs_defer_ops_continue to sort the inodes Added commentary about which inodes are locked
xfs: Hold inode locks in xfs_trans_alloc_dir
New patch
xfs: add parent pointer support to attribute code
Typo fix
xfs: extend transaction reservations for parent attribute
Made xfs_calc_parent_ptr_reservations static Whitespace fixes
xfs: parent pointer attribute creation
Fixed SPDX License Headers
Updated xfs_sb_version_hasparent to xfs_has_parent
Type def conversions
Whitespace
Added helper functions: xfs_parent_init, xfs_parent_defer_add,
xfs_parent_cancel
Investigated mount option that overrides larp option:
The larp global itself isnt used with in the delayed ops machinery due to race conditions with the syscall being toggled. Pptrs dont toggle so we can just set XFS_DA_OP_LOGGED to log the pptr with out requireing all attrs be logged
xfs: add parent attributes to link
Rebase to use new helpers: xfs_parent_init, xfs_parent_defer_add, xfs_parent_cancel, xfs_has_parent
xfs: add parent attributes to unlink
rebase to use new helpers
added helper function xfs_parent_defer_remove
xfs: Add parent pointers to rename
added extra parent remove for target_ip unlinks
rebase to use new helpers
xfs: Add the parent pointer support to the superblock version 5. changed XFS_SB_FEAT_RO_COMPAT_PARENT to XFS_SB_FEAT_INCOMPAT_PARENT
xfs: Add parent pointer ioctl
Added pi_parents[] to struct xfs_parent_ptr
Changed pptr helper defines into inline functions
White space/indentation
Reordered flag check in xfs_ioc_get_parent_pointer to be before the alloc Added gen number check in xfs_ioc_get_parent_pointer
Init args in loop body of xfs_attr_get_parent_pointer
Questions comments and feedback appreciated!
Thanks all!
Allison
Allison Henderson (18):
xfs: Fix multi-transaction larp replay
xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
xfs: Hold inode locks in xfs_ialloc
xfs: Hold inode locks in xfs_trans_alloc_dir
xfs: get directory offset when adding directory name
xfs: get directory offset when removing directory name
xfs: get directory offset when replacing a directory name
xfs: add parent pointer support to attribute code
xfs: define parent pointer xattr format
xfs: Add xfs_verify_pptr
xfs: extend transaction reservations for parent attributes
xfs: parent pointer attribute creation
xfs: add parent attributes to link
xfs: remove parent pointers in unlink
xfs: Add parent pointers to rename
xfs: Add the parent pointer support to the superblock version 5.
xfs: Add helper function xfs_attr_list_context_init
xfs: Add parent pointer ioctl
fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_attr.c | 53 ++++++- fs/xfs/libxfs/xfs_attr.h | 8 +- fs/xfs/libxfs/xfs_da_btree.h | 1 + fs/xfs/libxfs/xfs_da_format.h | 30 +++- fs/xfs/libxfs/xfs_defer.c | 28 +++- fs/xfs/libxfs/xfs_defer.h | 8 +- fs/xfs/libxfs/xfs_dir2.c | 21 ++- fs/xfs/libxfs/xfs_dir2.h | 7 +- fs/xfs/libxfs/xfs_dir2_block.c | 9 +- fs/xfs/libxfs/xfs_dir2_leaf.c | 8 +- fs/xfs/libxfs/xfs_dir2_node.c | 8 +- fs/xfs/libxfs/xfs_dir2_sf.c | 6 + fs/xfs/libxfs/xfs_format.h | 4 +- fs/xfs/libxfs/xfs_fs.h | 58 +++++++ fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/libxfs/xfs_parent.c | 159 +++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 39 +++++ fs/xfs/libxfs/xfs_sb.c | 4 + fs/xfs/libxfs/xfs_trans_resv.c | 105 ++++++++++--- fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_attr_item.c | 41 ++--- fs/xfs/xfs_attr_list.c | 17 ++- fs/xfs/xfs_file.c | 1 + fs/xfs/xfs_inode.c | 271 +++++++++++++++++++++++++-------- fs/xfs/xfs_inode.h | 1 + fs/xfs/xfs_ioctl.c | 149 +++++++++++++++--- fs/xfs/xfs_ioctl.h | 2 + fs/xfs/xfs_ondisk.h | 4 + fs/xfs/xfs_parent_utils.c | 134 ++++++++++++++++ fs/xfs/xfs_parent_utils.h | 22 +++ fs/xfs/xfs_qm.c | 4 +- fs/xfs/xfs_super.c | 4 + fs/xfs/xfs_symlink.c | 6 +- fs/xfs/xfs_trans.c | 6 +- fs/xfs/xfs_xattr.c | 2 +- fs/xfs/xfs_xattr.h | 1 + 37 files changed, 1056 insertions(+), 170 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h
Signature · 1 line
2.25.1
PATCH RESEND v2 1/18 xfs: Fix multi-transaction larp replay
Recent parent pointer testing has exposed a bug in the underlying attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more to do. This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a chance to finish. This can lead to interleaved set and remove operations that may clobber the attribute fork. Fix this by deferring all work for any attribute operation.
--- fs/xfs/xfs_attr_item.c | 35 ++++++++--------------------------- 1 file changed, 8 insertions(+), 27 deletions(-)
fs/xfs/xfs_attr_item.c +8 -27
@@ -635,52 +635,33 @@ xfs_attri_item_recover(break ;case XFS_ATTRI_OP_FLAGS_REMOVE :if (!xfs_inode_hasattr (args->dp)) - goto out; +return 0 ; attr->xattri_dela_state =xfs_attr_init_remove_state (args);break ;default :ASSERT (0 ); - error = -EFSCORRUPTED ; - goto out; +return -EFSCORRUPTED ; }xfs_init_attr_trans (args, &tres, &total); error =xfs_trans_alloc (mp, &tres, total,0 ,XFS_TRANS_RESERVE , &tp);if (error) - goto out; +return error; args->trans = tp; done_item =xfs_trans_get_attrd (tp, attrip); + args->trans->t_flags |=XFS_TRANS_HAS_INTENT_DONE ; +set_bit (XFS_LI_DIRTY , &done_item->attrd_item.li_flags);xfs_ilock (ip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, ip,0 ); - error =xfs_xattri_finish_update (attr, done_item); -if (error == -EAGAIN ) { -/* -* There's more work to do, so add the intent item to this -* transaction so that we can continue it later. -*/ -xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list); - error =xfs_defer_ops_capture_and_commit (tp, capture_list); -if (error) - goto out_unlock; - -xfs_iunlock (ip,XFS_ILOCK_EXCL ); -xfs_irele (ip); -return 0 ; - } -if (error) { -xfs_trans_cancel (tp); - goto out_unlock; - } - +xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list); error =xfs_defer_ops_capture_and_commit (tp, capture_list); -out_unlock: +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_irele (ip); -out: -xfs_attr_free_item (attr); +return error; }
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 58 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more to do. This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a chance to finish. This can lead to interleaved set and remove operations that may clobber the attribute fork. Fix this by deferring all work for any attribute operation.
Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_attr_item.c | 35 ++++++++--------------------------- 1 file changed, 8 insertions(+), 27 deletions(-)fs/xfs/xfs_attr_item.c +5
-15@@ -635,52 +635,33 @@ xfs_attri_item_recover(break ;case XFS_ATTRI_OP_FLAGS_REMOVE :if (!xfs_inode_hasattr (args->dp)) - goto out; +return 0 ; attr->xattri_dela_state =xfs_attr_init_remove_state (args);break ;default :ASSERT (0 ); - error = -EFSCORRUPTED ; - goto out; +return -EFSCORRUPTED ; }xfs_init_attr_trans (args, &tres, &total); error =xfs_trans_alloc (mp, &tres, total,0 ,XFS_TRANS_RESERVE , &tp);if (error) - goto out; +return error; args->trans = tp; done_item =xfs_trans_get_attrd (tp, attrip); + args->trans->t_flags |=XFS_TRANS_HAS_INTENT_DONE ; +set_bit (XFS_LI_DIRTY , &done_item->attrd_item.li_flags);xfs_ilock (ip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, ip,0 ); - error =xfs_xattri_finish_update (attr, done_item); -if (error == -EAGAIN ) { -/* -* There's more work to do, so add the intent item to this -* transaction so that we can continue it later. -*/ -xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list); - error =xfs_defer_ops_capture_and_commit (tp, capture_list); -if (error) - goto out_unlock; -
fs/xfs/xfs_attr_item.c xfs_attri_item_recover( · line 659
Quoted 10 lines
fs/xfs/xfs_attr_item.c +1
-9-xfs_iunlock (ip,XFS_ILOCK_EXCL ); -xfs_irele (ip); -return 0 ; - } -if (error) { -xfs_trans_cancel (tp); - goto out_unlock; - } - +xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list);
This seems a little convoluted to me. Maybe? Maybe not?
1. Log recovery recreates an incore xfs_attri_log_item from what it finds in the log.
2. This function then logs an xattrd for the recovered xattri item.
3. Then it creates a new xfs_attr_intent to complete the operation.
4. Finally, it calls xfs_defer_ops_capture_and_commit, which logs a new xattri for the intent created in step 3 and also commits the xattrd for the first xattri.
IOWs, the only difference between before and after is that we're not advancing one more step through the state machine as part of log recovery. From the perspective of the log, the recovery function merely replaces the recovered xattri log item with a new one.
Why can't we just attach the recovered xattri to the xfs_defer_pending that is created to point to the xfs_attr_intent that's created in step 3, and skip the xattrd?
I /think/ the answer to that question is that we might need to move the log tail forward to free enough log space to finish the intent items, so creating the extra xattrd/xattri (a) avoid the complexity of submitting an incore intent item *and* a log intent item to the defer ops machinery; and (b) avoid livelocks in log recovery. Therefore, we actually need to do it this way.
IOWS, I *think* this is ok, but want to see if others have differing perspectives on how log item recovery works?
--D
Quoted 14 lines
fs/xfs/xfs_attr_item.c +2
-3error =xfs_defer_ops_capture_and_commit (tp, capture_list); -out_unlock: +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_irele (ip); -out: -xfs_attr_free_item (attr); +return error; }Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more to do.
Yup.
Quoted 3 lines
Quoted 3 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a chance to finish.
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log recovery, right?
Quoted 3 lines
Quoted 3 lines
This can lead to interleaved set and remove operations that may clobber the attribute fork. Fix this by deferring all work for any attribute operation.
Which means this should be an impossible situation.
That is, if we crash before the final attrd DONE intent is written to the log, it means that new attr intents for modifications made *after* the current attr modification was completed will not be present in the log. We have strict ordering of committed operations in the journal, hence an operation on an inode has an incomplete intent *must* be the last operation and the *only* incomplete intent that is found in the journal for that inode.
Hence from an operational ordering persepective, this explanation for issue being seen doesn't make any sense to me. If there are multiple incomplete attri intents then we've either got a runtime journalling problem (a white-out issue? failing to relog the inode in each new intent?) or a log recovery problem (failing to match intent-done pairs correctly?), not a recovery deferral issue.
Hence I think we're still looking for the root cause of this problem...
Quoted 73 lines
Quoted 49 lines
Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_attr_item.c | 35 ++++++++--------------------------- 1 file changed, 8 insertions(+), 27 deletions(-)fs/xfs/xfs_attr_item.c +5
-15@@ -635,52 +635,33 @@ xfs_attri_item_recover(break ;case XFS_ATTRI_OP_FLAGS_REMOVE :if (!xfs_inode_hasattr (args->dp)) - goto out; +return 0 ; attr->xattri_dela_state =xfs_attr_init_remove_state (args);break ;default :ASSERT (0 ); - error = -EFSCORRUPTED ; - goto out; +return -EFSCORRUPTED ; }xfs_init_attr_trans (args, &tres, &total); error =xfs_trans_alloc (mp, &tres, total,0 ,XFS_TRANS_RESERVE , &tp);if (error) - goto out; +return error; args->trans = tp; done_item =xfs_trans_get_attrd (tp, attrip); + args->trans->t_flags |=XFS_TRANS_HAS_INTENT_DONE ; +set_bit (XFS_LI_DIRTY , &done_item->attrd_item.li_flags);xfs_ilock (ip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, ip,0 ); - error =xfs_xattri_finish_update (attr, done_item); -if (error == -EAGAIN ) { -/* -* There's more work to do, so add the intent item to this -* transaction so that we can continue it later. -*/ -xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list); - error =xfs_defer_ops_capture_and_commit (tp, capture_list); -if (error) - goto out_unlock; -fs/xfs/xfs_attr_item.c xfs_attri_item_recover( · line 659
Quoted 10 lines
fs/xfs/xfs_attr_item.c +1
-9-xfs_iunlock (ip,XFS_ILOCK_EXCL ); -xfs_irele (ip); -return 0 ; - } -if (error) { -xfs_trans_cancel (tp); - goto out_unlock; - } - +xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list);This seems a little convoluted to me. Maybe? Maybe not?
1. Log recovery recreates an incore
xfs_attri_log_itemfrom what it finds in the log.2. This function then logs an xattrd for the recovered xattri item.
3. Then it creates a new
xfs_attr_intentto complete the operation.4. Finally, it calls
xfs_defer_ops_capture_and_commit, which logs a new xattri for the intent created in step 3 and also commits the xattrd for the first xattri.
Quoted 8 lines
IOWs, the only difference between before and after is that we're not advancing one more step through the state machine as part of log recovery. From the perspective of the log, the recovery function merely replaces the recovered xattri log item with a new one.
Why can't we just attach the recovered xattri to the
xfs_defer_pendingthat is created to point to thexfs_attr_intentthat's created in step 3, and skip the xattrd?
Remember that attribute intents are different to all other intent types that we have. The existing extent based intents define a single indepedent operation that needs to be performed, and each step of the intent chain is completely independent of the previous step in the chain. e.g. removing the extent from the rmap btree is completely independent of removing it from the inode bmap btree - all that matters is that the removal from the bmbt happens first. The rmapbt removal can happen at any time after than, and is completely independent of any other bmbt or rmapbt operation. Similarly, the EFI can processed independently of all bmapbt and rmapbt modifications, it just has to happen after those modifications are done.
Hence if we crash during recovery, we can just restart from where-ever we got to in the middle of the intent chains and not have to care at all. IOWs, eventual consistency works with these chains because there is no dependencies between each step of the intent chain and each step is completely independent of the other steps.
Attribute intent chains are completely different. They link steps in a state machine together in a non-trivial, highly dependent chain. We can't just restart the chain in the middle like we can for the BUI->RUI->CUI->EFI chain because the on-disk attribute is in an unknown state and recovering that exact state is .... complex.
Hence the the first step of recovery is to return the attribute we are trying to modify back to a known state. That means we have to perform a removal of any existing attribute under that name first. Hence this first step should be replacing the existing attr intent with the intent that defines the recovery operation we are going to perform.
That means we need to translate set to replace so that cleanup is run first, replace needs to clean up the attr under that name regardless of whether it has the incomplete bit set on it or not. Remove is the only operation that runs the same as at runtime, as cleanup for remove is just repeating the remove operation from scratch.
Quoted 6 lines
I /think/ the answer to that question is that we might need to move the log tail forward to free enough log space to finish the intent items, so creating the extra xattrd/xattri (a) avoid the complexity of submitting an incore intent item *and* a log intent item to the defer ops machinery; and (b) avoid livelocks in log recovery. Therefore, we actually need to do it this way.
We really need the initial operation to rewrite the intent to match the recovery operation we are going to perform. Everything else is secondary.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 9 lines · Dave Chinner
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more to do.
Yup.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a chance to finish.
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log recovery, right?
No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the old one. It cant be an attr replace because technically the names are different.
So the recovered set grows the leaf, and returns the egain, then rest gets capture committed. Next up is the recovered remove which pulls out the fork, which causes problems when the rest of the set operation resumes as a deferred operation. Here is the link to the original discussion, it was quite a while ago:
https://lore.kernel.org/all/Yrzw9F5aGsaldrmR@magnolia/
I hope that helps?
Allison
Quoted 174 lines
Quoted 4 lines
Quoted 4 lines
This can lead to interleaved set and remove operations that may clobber the attribute fork. Fix this by deferring all work for any attribute operation.
Which means this should be an impossible situation.
That is, if we crash before the final attrd DONE intent is written to the log, it means that new attr intents for modifications made *after* the current attr modification was completed will not be present in the log. We have strict ordering of committed operations in the journal, hence an operation on an inode has an incomplete intent *must* be the last operation and the *only* incomplete intent that is found in the journal for that inode.
Hence from an operational ordering persepective, this explanation for issue being seen doesn't make any sense to me. If there are multiple incomplete attri intents then we've either got a runtime journalling problem (a white-out issue? failing to relog the inode in each new intent?) or a log recovery problem (failing to match intent-done pairs correctly?), not a recovery deferral issue.
Hence I think we're still looking for the root cause of this problem...
Quoted 87 lines
Quoted 54 lines
Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_attr_item.c | 35 ++++++++--------------------------- 1 file changed, 8 insertions(+), 27 deletions(-)fs/xfs/xfs_attr_item.c +5
-8@@ -635,52 +635,33 @@ xfs_attri_item_recover(break ;case XFS_ATTRI_OP_FLAGS_REMOVE :if (!xfs_inode_hasattr (args->dp)) - goto out; +return 0 ; attr->xattri_dela_state =xfs_attr_init_remove_state (args);break ;default :ASSERT (0 ); - error = -EFSCORRUPTED ; - goto out; +return -EFSCORRUPTED ; }xfs_init_attr_trans (args, &tres, &total); error =xfs_trans_alloc (mp, &tres, total,0 ,XFS_TRANS_RESERVE , &tp);if (error) - goto out; +return error; args->trans = tp; done_item =xfs_trans_get_attrd (tp, attrip); + args->trans->t_flags |=XFS_TRANS_HAS_INTENT_DONE ; +set_bit (XFS_LI_DIRTY , &done_item->attrd_item.li_flags);xfs_ilock (ip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, ip,0 ); - error =xfs_xattri_finish_update (attr, done_item); -if (error == -EAGAIN ) { -/* -* There's more work to do, so add the intent item to this
fs/xfs/xfs_attr_item.c +0
-3-* transaction so that we can continue it later. -*/ - xfs_defer_add(tp, XFS_DEFER_OPS_TYPE_ATTR, &attr-Quoted 1 line
xattri_list);fs/xfs/xfs_attr_item.c +0
-1- error = xfs_defer_ops_capture_and_commit(tp,
capture_list);fs/xfs/xfs_attr_item.c +0
-3-if (error) - goto out_unlock; -fs/xfs/xfs_attr_item.c xfs_attri_item_recover( · line 659
Quoted 10 lines
fs/xfs/xfs_attr_item.c +1
-9-xfs_iunlock (ip,XFS_ILOCK_EXCL ); -xfs_irele (ip); -return 0 ; - } -if (error) { -xfs_trans_cancel (tp); - goto out_unlock; - } - +xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list);This seems a little convoluted to me. Maybe? Maybe not?
1. Log recovery recreates an incore
xfs_attri_log_itemfrom what it finds in the log.2. This function then logs an xattrd for the recovered xattri item.
3. Then it creates a new
xfs_attr_intentto complete the operation.4. Finally, it calls
xfs_defer_ops_capture_and_commit, which logs a new
xattri for the intent created in step 3 and also commits the xattrd for
the first xattri.IOWs, the only difference between before and after is that we're not
advancing one more step through the state machine as part of log recovery. From the perspective of the log, the recovery function merely
replaces the recovered xattri log item with a new one.Quoted 5 lines
Why can't we just attach the recovered xattri to the
xfs_defer_pending
that is created to point to thexfs_attr_intentthat's created in step
3, and skip the xattrd?Remember that attribute intents are different to all other intent types that we have. The existing extent based intents define a single indepedent operation that needs to be performed, and each step of the intent chain is completely independent of the previous step in the chain. e.g. removing the extent from the rmap btree is completely independent of removing it from the inode bmap btree - all that matters is that the removal from the bmbt happens first. The rmapbt removal can happen at any time after than, and is completely independent of any other bmbt or rmapbt operation. Similarly, the EFI can processed independently of all bmapbt and rmapbt modifications, it just has to happen after those modifications are done.
Hence if we crash during recovery, we can just restart from where-ever we got to in the middle of the intent chains and not have to care at all. IOWs, eventual consistency works with these chains because there is no dependencies between each step of the intent chain and each step is completely independent of the other steps.
Attribute intent chains are completely different. They link steps in a state machine together in a non-trivial, highly dependent chain. We can't just restart the chain in the middle like we can for the
BUI->RUI->CUI->EFIchain because the on-disk attribute is in an unknown state and recovering that exact state is .... complex.Hence the the first step of recovery is to return the attribute we are trying to modify back to a known state. That means we have to perform a removal of any existing attribute under that name first. Hence this first step should be replacing the existing attr intent with the intent that defines the recovery operation we are going to perform.
That means we need to translate set to replace so that cleanup is run first, replace needs to clean up the attr under that name regardless of whether it has the incomplete bit set on it or not. Remove is the only operation that runs the same as at runtime, as cleanup for remove is just repeating the remove operation from scratch.
Quoted 9 lines
I /think/ the answer to that question is that we might need to move the
log tail forward to free enough log space to finish the intent items, so
creating the extra xattrd/xattri (a) avoid the complexity of submitting
an incore intent item *and* a log intent item to the defer ops machinery; and (b) avoid livelocks in log recovery. Therefore, we actually need to do it this way.We really need the initial operation to rewrite the intent to match the recovery operation we are going to perform. Everything else is secondary.
Cheers,
Dave.
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 14 lines · Alli
Quoted 13 lines · Dave Chinner
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more to do.
Yup.
Quoted 3 lines
Quoted 3 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a chance to finish.
Quoted 6 lines
Quoted 3 lines
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log recovery, right?
No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the old one.
Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is a pretty important design rule change...
The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.
One of the reasons I rewrote the attr state machine for LARP was to enable new multiple attr operation chains to be easily build from the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent intent chains for each modification.
Quoted 2 lines
It cant be an attr replace because technically the names are different.
I disagree - we have all the pieces we need in the state machine already, we just need to define separate attr names for the remove and insert steps in the attr intent.
That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation, but we actually implement it as a "replace entire attribute" operation.
Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name to exist in the attr tree at the same time. IOWs, the attr value overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always have either the old or new xattr visible.
With LARP, we can remove the original attr first, thereby avoiding the need for two versions of the xattr to exist in the tree in the first place. However, we have to do these two operations as a pair of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr that we are overwriting in the intent. Hence we can always recover the modification to completion no matter where in the operation we fail.
When it comes to a parent attr rename operation, we are effectively doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the same sequence as a "replace value" operation, except for the fact that the new attr we add has a different name.
Hence the only real difference between the existing "attr replace" and the intent chain we need for "parent attr rename" is that we have to log two attr names instead of one. Basically, we have a new XFS_ATTRI_OP_FLAGS... type for this, and that's what tells us that we are operating on two different attributes instead of just one.
The recovery operation becomes slightly different - we have to run a remove on the old, then a replace on the new - so there a little bit of new code needed to manage that in the state machine.
These, however, are just small tweaks on the existing replace attr operation, and there should be little difference in performance or overhead between a "replace value" and a "replace entire xattr" operation as they are largely the same runtime operation for LARP.
Quoted 4 lines
So the recovered set grows the leaf, and returns the egain, then rest gets capture committed. Next up is the recovered remove which pulls out the fork, which causes problems when the rest of the set operation resumes as a deferred operation.
Yup, and all this goes away when we build the right intent chain for replacing a parent attr rename....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 27 lines · Dave Chinner
Quoted 14 lines · Alli
Quoted 13 lines · Dave Chinner
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more to do.
Yup.
Quoted 3 lines
Quoted 3 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a chance to finish.
Quoted 6 lines
Quoted 3 lines
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log recovery, right?
No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the old one.
Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is a pretty important design rule change...
Quoted 6 lines
The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.
Back when I made the suggestion that resulted in this patch, I was pondering why it is that (say) atomic swapext didn't suffer from these recovery problems, and I realized that for any given inode, you can only have one ongoing swapext operation at a time. That's why recovery of swapext operations works fine, whereas pptr recovery has this quirk.
At the time, my thought process was more narrowly focused on making log recovery mimic runtime more closely. I didn't make the connection between this problem and the other open question I had (see the bottom) about how to fix pptr attrs when rebuilding a directory.
Quoted 9 lines
One of the reasons I rewrote the attr state machine for LARP was to enable new multiple attr operation chains to be easily build from the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent intent chains for each modification.
Quoted 2 lines
It cant be an attr replace because technically the names are different.
Quoted 8 lines
I disagree - we have all the pieces we need in the state machine already, we just need to define separate attr names for the remove and insert steps in the attr intent.
That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation, but we actually implement it as a "replace entire attribute" operation.
OH. Right. I forgot that ATTR_REPLACE=="replace entire attr".
If I'm understanding this right, that means that the xfs_rename patch ought to detect the situation where there's an existing dirent in the target directory, and do something along the lines of:
} else { /* target_ip != NULL */
xfs_dir_replace(...);
xfs_parent_defer_replace(tp, new_parent_ptr, target_dp,
old_diroffset, target_name,
new_diroffset);
xfs_trans_ichgtime(...);Where the xfs_parent_defer_replace operation does an ATTR_REPLACE to switch:
(target_dp_ino, target_gen, old_diroffset) == <dontcare>
to this:
(target_dp_ino, target_gen, new_diroffset) == target_name
except, I think we have to log the old name in addition to the new name, because userspace ATTR_REPLACE operations don't allow name changes?
I guess this also implies that xfs_dir_replace will pass out the offset of the old name, in addition to the offset of the new name.
Quoted 22 lines
Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name to exist in the attr tree at the same time. IOWs, the attr value overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always have either the old or new xattr visible.
With LARP, we can remove the original attr first, thereby avoiding the need for two versions of the xattr to exist in the tree in the first place. However, we have to do these two operations as a pair of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr that we are overwriting in the intent. Hence we can always recover the modification to completion no matter where in the operation we fail.
When it comes to a parent attr rename operation, we are effectively doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the same sequence as a "replace value" operation, except for the fact that the new attr we add has a different name.
Quoted 5 lines
Hence the only real difference between the existing "attr replace" and the intent chain we need for "parent attr rename" is that we have to log two attr names instead of one. Basically, we have a new
XFS_ATTRI_OP_FLAGS... type for this, and that's what tells us that we are operating on two different attributes instead of just one.
This answers my earlier question: Yes, and yes.
Quoted 9 lines
The recovery operation becomes slightly different - we have to run a remove on the old, then a replace on the new - so there a little bit of new code needed to manage that in the state machine.
These, however, are just small tweaks on the existing replace attr operation, and there should be little difference in performance or overhead between a "replace value" and a "replace entire xattr" operation as they are largely the same runtime operation for LARP.
Quoted 7 lines
Quoted 4 lines
So the recovered set grows the leaf, and returns the egain, then rest gets capture committed. Next up is the recovered remove which pulls out the fork, which causes problems when the rest of the set operation resumes as a deferred operation.
Yup, and all this goes away when we build the right intent chain for replacing a parent attr rename....
Funnily enough, just last week I had thought that online repair was going to require the ability to replace an entire xattr...
https://djwong.org/docs/xfs-online-fsck-design/#parent-pointers
--D
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 132 lines · Darrick J. Wong
Quoted 37 lines · Dave Chinner
Quoted 25 lines · Alli
Quoted 19 lines · Dave Chinner
Quoted 9 lines · Darrick J. Wong
Quoted 7 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying
attr replay. A multi transaction replay currently performs a
single step of the replay, then deferrs the rest if there is more
to do.Yup.
Quoted 4 lines
Quoted 4 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a
chance to finish.Quoted 5 lines
What other attr replays are we racing against? There can only be
one incomplete attr item intent/done chain per inode present in log
recovery, right?Quoted 4 lines
No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is
a pretty important design rule change...Quoted 6 lines
The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.
Back when I made the suggestion that resulted in this patch, I was pondering why it is that (say) atomic swapext didn't suffer from these
recovery problems, and I realized that for any given inode, you can only
have one ongoing swapext operation at a time. That's why recovery of swapext operations works fine, whereas pptr recovery has this quirk.At the time, my thought process was more narrowly focused on making log
recovery mimic runtime more closely. I didn't make the connection between this problem and the other open question I had (see the bottom)
about how to fix pptr attrs when rebuilding a directory.Quoted 9 lines
One of the reasons I rewrote the attr state machine for LARP was to enable new multiple attr operation chains to be easily build from the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent intent chains for each modification.
Quoted 2 lines
It cant be an attr replace because technically the names are different.
Quoted 8 lines
I disagree - we have all the pieces we need in the state machine already, we just need to define separate attr names for the remove and insert steps in the attr intent.
That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation, but we actually implement it as a "replace entire attribute" operation.
OH. Right. I forgot that ATTR_REPLACE=="replace entire attr".
If I'm understanding this right, that means that the
xfs_renamepatch ought to detect the situation where there's an existing dirent in the target directory, and do something along the lines of:} else { /* target_ip != NULL */ xfs_dir_replace(...); xfs_parent_defer_replace(tp, new_parent_ptr, target_dp, old_diroffset, target_name, new_diroffset); xfs_trans_ichgtime(...);Where the
xfs_parent_defer_replaceoperation does anATTR_REPLACEto switch:(
target_dp_ino,target_gen,old_diroffset) == <dontcare>to this:
(
target_dp_ino,target_gen,new_diroffset) ==target_nameexcept, I think we have to log the old name in addition to the new name,
because userspaceATTR_REPLACEoperations don't allow name changes?I guess this also implies that
xfs_dir_replacewill pass out the offset
of the old name, in addition to the offset of the new name.Quoted 22 lines
Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name to exist in the attr tree at the same time. IOWs, the attr value overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always have either the old or new xattr visible.
With LARP, we can remove the original attr first, thereby avoiding the need for two versions of the xattr to exist in the tree in the first place. However, we have to do these two operations as a pair of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr that we are overwriting in the intent. Hence we can always recover the modification to completion no matter where in the operation we fail.
When it comes to a parent attr rename operation, we are effectively doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the same sequence as a "replace value" operation, except for the fact that the new attr we add has a different name.
Quoted 7 lines
Quoted 5 lines
Hence the only real difference between the existing "attr replace" and the intent chain we need for "parent attr rename" is that we have to log two attr names instead of one. Basically, we have a new
XFS_ATTRI_OP_FLAGS... type for this, and that's what tells us that we are operating on two different attributes instead of just one.This answers my earlier question: Yes, and yes.
I see, alrighty then, I'll see if I can put together a new XFS_ATTRI_OP_FLAGS type that carries both the old and new name. That sounds like it should work. Thanks for all the feed back!
Allison
Quoted 37 lines
Quoted 20 lines
The recovery operation becomes slightly different - we have to run a
remove on the old, then a replace on the new - so there a little bit
of new code needed to manage that in the state machine.These, however, are just small tweaks on the existing replace attr operation, and there should be little difference in performance or overhead between a "replace value" and a "replace entire xattr" operation as they are largely the same runtime operation for LARP.
Quoted 7 lines
So the recovered set grows the leaf, and returns the egain, then rest
gets capture committed. Next up is the recovered remove which pulls
out the fork, which causes problems when the rest of the set operation
resumes as a deferred operation.Quoted 3 lines
Yup, and all this goes away when we build the right intent chain for
replacing a parent attr rename....Funnily enough, just last week I had thought that online repair was going to require the ability to replace an entire xattr...
--D
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
In reply to Dave ChinnerQuoted 80 lines · Dave Chinner
Quoted 18 lines · Alli
Quoted 17 lines · Dave Chinner
Quoted 8 lines · Darrick J. Wong
Quoted 6 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying
attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more
to do.Yup.
Quoted 4 lines
Quoted 4 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a
chance to finish.Quoted 8 lines
Quoted 4 lines
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log
recovery, right?No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is a pretty important design rule change...
The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.
One of the reasons I rewrote the attr state machine for LARP was to enable new multiple attr operation chains to be easily build from the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent intent chains for each modification.
Quoted 2 lines
It cant be an attr replace because technically the names are different.
I disagree - we have all the pieces we need in the state machine already, we just need to define separate attr names for the remove and insert steps in the attr intent.
That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation, but we actually implement it as a "replace entire attribute" operation.
Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name to exist in the attr tree at the same time. IOWs, the attr value overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always have either the old or new xattr visible.
With LARP, we can remove the original attr first, thereby avoiding the need for two versions of the xattr to exist in the tree in the first place. However, we have to do these two operations as a pair of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr that we are overwriting in the intent. Hence we can always recover the modification to completion no matter where in the operation we fail.
When it comes to a parent attr rename operation, we are effectively doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the same sequence as a "replace value" operation, except for the fact that the new attr we add has a different name.
Quoted 3 lines
Hence the only real difference between the existing "attr replace" and the intent chain we need for "parent attr rename" is that we have to log two attr names instead of one.
To be clear, this would imply expanding xfs_attri_log_format to have another alfi_new_name_len feild and another iovec for the attr intent right? Does that cause issues to change the on disk log layout after the original has merged? Or is that ok for things that are still experimental? Thanks!
Allison
Quoted 27 lines
Basically, we have a new
XFS_ATTRI_OP_FLAGS... type for this, and that's what tells us that we are operating on two different attributes instead of just one.The recovery operation becomes slightly different - we have to run a remove on the old, then a replace on the new - so there a little bit of new code needed to manage that in the state machine.
These, however, are just small tweaks on the existing replace attr operation, and there should be little difference in performance or overhead between a "replace value" and a "replace entire xattr" operation as they are largely the same runtime operation for LARP.
Quoted 7 lines
So the recovered set grows the leaf, and returns the egain, then rest
gets capture committed. Next up is the recovered remove which pulls
out the fork, which causes problems when the rest of the set operation
resumes as a deferred operation.Yup, and all this goes away when we build the right intent chain for replacing a parent attr rename....
Cheers,
Dave.
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 85 lines · Alli
Quoted 80 lines · Dave Chinner
Quoted 18 lines · Alli
Quoted 17 lines · Dave Chinner
Quoted 8 lines · Darrick J. Wong
Quoted 6 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying
attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more
to do.Yup.
Quoted 4 lines
Quoted 4 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a
chance to finish.Quoted 8 lines
Quoted 4 lines
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log
recovery, right?No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is a pretty important design rule change...
The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.
One of the reasons I rewrote the attr state machine for LARP was to enable new multiple attr operation chains to be easily build from the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent intent chains for each modification.
Quoted 2 lines
It cant be an attr replace because technically the names are different.
I disagree - we have all the pieces we need in the state machine already, we just need to define separate attr names for the remove and insert steps in the attr intent.
That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation, but we actually implement it as a "replace entire attribute" operation.
Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name to exist in the attr tree at the same time. IOWs, the attr value overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always have either the old or new xattr visible.
With LARP, we can remove the original attr first, thereby avoiding the need for two versions of the xattr to exist in the tree in the first place. However, we have to do these two operations as a pair of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr that we are overwriting in the intent. Hence we can always recover the modification to completion no matter where in the operation we fail.
When it comes to a parent attr rename operation, we are effectively doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the same sequence as a "replace value" operation, except for the fact that the new attr we add has a different name.Quoted 3 lines
Hence the only real difference between the existing "attr replace" and the intent chain we need for "parent attr rename" is that we have to log two attr names instead of one.
Quoted 5 lines
To be clear, this would imply expanding
xfs_attri_log_formatto have anotheralfi_new_name_lenfeild and another iovec for the attr intent right? Does that cause issues to change the on disk log layout after the original has merged? Or is that ok for things that are still experimental? Thanks!
XFS_SB_FEAT_INCOMPAT_PARENT should protect against that, since the userspace xattr api does not support replacing an attr's name, only the value.
--D
Quoted 30 lines
Allison
Quoted 27 lines
Basically, we have a new
XFS_ATTRI_OP_FLAGS... type for this, and that's what tells us that we are operating on two different attributes instead of just one.The recovery operation becomes slightly different - we have to run a remove on the old, then a replace on the new - so there a little bit of new code needed to manage that in the state machine.
These, however, are just small tweaks on the existing replace attr operation, and there should be little difference in performance or overhead between a "replace value" and a "replace entire xattr" operation as they are largely the same runtime operation for LARP.
Quoted 7 lines
So the recovered set grows the leaf, and returns the egain, then rest
gets capture committed. Next up is the recovered remove which pulls
out the fork, which causes problems when the rest of the set operation
resumes as a deferred operation.Yup, and all this goes away when we build the right intent chain for replacing a parent attr rename....
Cheers,
Dave.
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
In reply to AlliQuoted 85 lines · Alli
Quoted 80 lines · Dave Chinner
Quoted 18 lines · Alli
Quoted 17 lines · Dave Chinner
Quoted 8 lines · Darrick J. Wong
Quoted 6 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying
attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more
to do.Yup.
Quoted 4 lines
Quoted 4 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a
chance to finish.Quoted 8 lines
Quoted 4 lines
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log
recovery, right?No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is a pretty important design rule change...
The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.
One of the reasons I rewrote the attr state machine for LARP was to enable new multiple attr operation chains to be easily build from the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent intent chains for each modification.
Quoted 2 lines
It cant be an attr replace because technically the names are different.
I disagree - we have all the pieces we need in the state machine already, we just need to define separate attr names for the remove and insert steps in the attr intent.
That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation, but we actually implement it as a "replace entire attribute" operation.
Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name to exist in the attr tree at the same time. IOWs, the attr value overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always have either the old or new xattr visible.
With LARP, we can remove the original attr first, thereby avoiding the need for two versions of the xattr to exist in the tree in the first place. However, we have to do these two operations as a pair of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr that we are overwriting in the intent. Hence we can always recover the modification to completion no matter where in the operation we fail.
When it comes to a parent attr rename operation, we are effectively doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the same sequence as a "replace value" operation, except for the fact that the new attr we add has a different name.Quoted 3 lines
Hence the only real difference between the existing "attr replace" and the intent chain we need for "parent attr rename" is that we have to log two attr names instead of one.
Quoted 5 lines
To be clear, this would imply expanding
xfs_attri_log_formatto have anotheralfi_new_name_lenfeild and another iovec for the attr intent right? Does that cause issues to change the on disk log layout after the original has merged? Or is that ok for things that are still experimental? Thanks!
I think we can get away with this quite easily without breaking the existing experimental code.
struct xfs_attri_log_format {
uint16_t alfi_type; /* attri log item type */
uint16_t alfi_size; /* size of this item */
uint32_t __pad; /* pad to 64 bit aligned */
uint64_t alfi_id; /* attri identifier */
uint64_t alfi_ino; /* the inode for this attr operation */
uint32_t alfi_op_flags; /* marks the op as a set or remove */
uint32_t alfi_name_len; /* attr name length */
uint32_t alfi_value_len; /* attr value length */
uint32_t alfi_attr_filter;/* attr filter flags */};
We have a padding field in there that is currently all zeros. Let's make that a count of the number of {name, value} tuples that are appended to the format. i.e.
struct xfs_attri_log_name {
uint32_t alfi_op_flags; /* marks the op as a set or remove */
uint32_t alfi_name_len; /* attr name length */
uint32_t alfi_value_len; /* attr value length */
uint32_t alfi_attr_filter;/* attr filter flags */};
struct xfs_attri_log_format {
uint16_t alfi_type; /* attri log item type */
uint16_t alfi_size; /* size of this item */
uint8_t alfi_attr_cnt; /* count of name/val pairs */
uint8_t __pad1; /* pad to 64 bit aligned */
uint16_t __pad2; /* pad to 64 bit aligned */
uint64_t alfi_id; /* attri identifier */
uint64_t alfi_ino; /* the inode for this attr operation */
struct xfs_attri_log_name alfi_attr[]; /* attrs to operate on */
};Basically, the size and shape of the structure has not changed, and if alfi_attr_cnt == 0 we just treat it as if alfi_attr_cnt == 1 as the backwards compat code for the existing code.
And then we just have as many followup regions for name/val pairs as are defined by the alfi_attr_cnt and alfi_attr[] parts of the structure. Each attr can have a different operation performed on them, and they can have different filters applied so they can exist in different namespaces, too.
SO I don't think we need a new on-disk feature bit for this enhancement - it definitely comes under the heading of "this stuff is experimental, this is the sort of early structure revision that EXPERIMENTAL is supposed to cover....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 139 lines · Dave Chinner
Quoted 85 lines · Alli
Quoted 80 lines · Dave Chinner
Quoted 18 lines · Alli
Quoted 17 lines · Dave Chinner
Quoted 8 lines · Darrick J. Wong
Quoted 6 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying
attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more
to do.Yup.
Quoted 4 lines
Quoted 4 lines
This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a
chance to finish.Quoted 8 lines
Quoted 4 lines
What other attr replays are we racing against? There can only be one incomplete attr item intent/done chain per inode present in log
recovery, right?No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is a pretty important design rule change...
The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.
One of the reasons I rewrote the attr state machine for LARP was to enable new multiple attr operation chains to be easily build from the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent intent chains for each modification.
Quoted 2 lines
It cant be an attr replace because technically the names are different.
I disagree - we have all the pieces we need in the state machine already, we just need to define separate attr names for the remove and insert steps in the attr intent.
That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation, but we actually implement it as a "replace entire attribute" operation.
Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name to exist in the attr tree at the same time. IOWs, the attr value overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always have either the old or new xattr visible.
With LARP, we can remove the original attr first, thereby avoiding the need for two versions of the xattr to exist in the tree in the first place. However, we have to do these two operations as a pair of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr that we are overwriting in the intent. Hence we can always recover the modification to completion no matter where in the operation we fail.
When it comes to a parent attr rename operation, we are effectively doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the same sequence as a "replace value" operation, except for the fact that the new attr we add has a different name.Quoted 3 lines
Hence the only real difference between the existing "attr replace" and the intent chain we need for "parent attr rename" is that we have to log two attr names instead of one.
Quoted 5 lines
To be clear, this would imply expanding
xfs_attri_log_formatto have anotheralfi_new_name_lenfeild and another iovec for the attr intent right? Does that cause issues to change the on disk log layout after the original has merged? Or is that ok for things that are still experimental? Thanks!I think we can get away with this quite easily without breaking the existing experimental code.
struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint32_t __pad; /* pad to 64 bit aligned */ uint64_t alfi_id; /* attri identifier */ uint64_t alfi_ino; /* the inode for this attr operation */ uint32_t alfi_op_flags; /* marks the op as a set or remove */ uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
We have a padding field in there that is currently all zeros. Let's make that a count of the number of {name, value} tuples that are appended to the format. i.e.
struct
xfs_attri_log_name{uint32_t alfi_op_flags; /* marks the op as a set or remove */ uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint8_t alfi_attr_cnt; /* count of name/val pairs */ uint8_t __pad1; /* pad to 64 bit aligned */ uint16_t __pad2; /* pad to 64 bit aligned */ uint64_t alfi_id; /* attri identifier */ uint64_t alfi_ino; /* the inode for this attr operation */ struct xfs_attri_log_name alfi_attr[]; /* attrs to operate on */ };Basically, the size and shape of the structure has not changed, and if
alfi_attr_cnt== 0 we just treat it as ifalfi_attr_cnt== 1 as the backwards compat code for the existing code.And then we just have as many followup regions for name/val pairs as are defined by the
alfi_attr_cntand alfi_attr[] parts of the structure. Each attr can have a different operation performed on them, and they can have different filters applied so they can exist in different namespaces, too.
Quoted 4 lines
SO I don't think we need a new on-disk feature bit for this enhancement - it definitely comes under the heading of "this stuff is experimental, this is the sort of early structure revision that EXPERIMENTAL is supposed to cover....
You might even callit "alfi_extra_names" to avoid the "0 means 1" stuff. ;)
--D
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 186 lines · Darrick J. Wong
Quoted 180 lines · Dave Chinner
Quoted 116 lines · Alli
Quoted 109 lines · Dave Chinner
Quoted 30 lines · Alli
Quoted 23 lines · Dave Chinner
Quoted 11 lines · Darrick J. Wong
Quoted 8 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying
attr replay. A multi transaction replay currently performs a
single step of the replay, then deferrs the rest if there is
more
to do.Yup.
Quoted 6 lines
Quoted 6 lines
This causes race conditions with other attr replays that
might be recovered before the remaining deferred work has had
a
chance to finish.Quoted 6 lines
What other attr replays are we racing against? There can only be
one incomplete attr item intent/done chain per inode present in
log
recovery, right?Quoted 5 lines
No, a rename queues up a set and remove before committing the transaction. One for the new parent pointer, and another to remove
the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is
a pretty important design rule change...The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent
lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.One of the reasons I rewrote the attr state machine for LARP was to
enable new multiple attr operation chains to be easily build from
the entry points the state machien provides. Parent attr rename needs a new intent chain to be built, not run multiple independent
intent chains for each modification.Quoted 2 lines
It cant be an attr replace because technically the names are different.
I disagree - we have all the pieces we need in the state machine
already, we just need to define separate attr names for the remove and insert steps in the attr intent.That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation,
but we actually implement it as a "replace entire attribute" operation.Without LARP, we do that overwrite in independent steps via an intermediate INCOMPLETE state to allow two xattrs of the same name
to exist in the attr tree at the same time. IOWs, the attr value
overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always
have either the old or new xattr visible.With LARP, we can remove the original attr first, thereby avoiding
the need for two versions of the xattr to exist in the tree in the
first place. However, we have to do these two operations as a pair
of linked independent operations. The intent chain provides the linking, and requires us to log the name and the value of the attr
that we are overwriting in the intent. Hence we can always recover
the modification to completion no matter where in the operation we
fail.When it comes to a parent attr rename operation, we are effectively
doing two linked operations - remove the old attr, set the new attr
- on different attributes. Implementation wise, it is exactly the
same sequence as a "replace value" operation, except for the fact
that the new attr we add has a different name.Quoted 5 lines
Hence the only real difference between the existing "attr replace"
and the intent chain we need for "parent attr rename" is that we
have to log two attr names instead of one.Quoted 8 lines
To be clear, this would imply expanding
xfs_attri_log_formatto have
anotheralfi_new_name_lenfeild and another iovec for the attr intent
right? Does that cause issues to change the on disk log layout after
the original has merged? Or is that ok for things that are still experimental? Thanks!I think we can get away with this quite easily without breaking the existing experimental code.
struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint32_t __pad; /* pad to 64 bit aligned */ uint64_t alfi_id; /* attri identifier */ uint64_t alfi_ino; /* the inode for this attr operation */ uint32_t alfi_op_flags; /* marks the op as a set or remove */ uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
We have a padding field in there that is currently all zeros. Let's make that a count of the number of {name, value} tuples that are appended to the format. i.e.
struct
xfs_attri_log_name{
uint32_talfi_op_flags; /* marks the op as a set or
remove */uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint8_t alfi_attr_cnt; /* count of name/val pairs */ uint8_t __pad1; /* pad to 64 bitaligned */
uint16_t__pad2; /* pad to 64 bit aligned */
uint64_talfi_id; /* attri identifier */
uint64_talfi_ino; /* the inode for this attr
operation */struct xfs_attri_log_name alfi_attr[]; /* attrs to operate on*/
};Basically, the size and shape of the structure has not changed, and if
alfi_attr_cnt== 0 we just treat it as ifalfi_attr_cnt== 1 as the backwards compat code for the existing code.And then we just have as many followup regions for name/val pairs as are defined by the
alfi_attr_cntand alfi_attr[] parts of the structure. Each attr can have a different operation performed on them, and they can have different filters applied so they can exist in different namespaces, too.Quoted 4 lines
SO I don't think we need a new on-disk feature bit for this enhancement - it definitely comes under the heading of "this stuff is experimental, this is the sort of early structure revision that EXPERIMENTAL is supposed to cover....
Quoted 5 lines
You might even callit "
alfi_extra_names" to avoid the "0 means 1" stuff.
;)--D
Oh, I just noticed these comments this morning when I sent out the new attri/d patch. I'll add this changes to v2. Please let me know if there's anything else you'd like me to change from the v1. Thx!
Allison
Quoted 7 lines
Quoted 7 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 215 lines · Alli
Quoted 214 lines · Darrick J. Wong
Quoted 202 lines · Dave Chinner
Quoted 126 lines · Alli
Quoted 119 lines · Dave Chinner
Quoted 31 lines · Alli
Quoted 24 lines · Dave Chinner
Quoted 12 lines · Darrick J. Wong
Quoted 9 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the
underlying
attr replay. A multi transaction replay currently performs a
single step of the replay, then deferrs the rest if there is
more
to do.Yup.
Quoted 6 lines
Quoted 6 lines
This causes race conditions with other attr replays that
might be recovered before the remaining deferred work has had
a
chance to finish.Quoted 6 lines
What other attr replays are we racing against? There can only be
one incomplete attr item intent/done chain per inode present in
log
recovery, right?Quoted 7 lines
No, a rename queues up a set and remove before committing the
transaction. One for the new parent pointer, and another to
remove
the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is
a pretty important design rule change...The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent
lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.One of the reasons I rewrote the attr state machine for LARP was to
enable new multiple attr operation chains to be easily build from
the entry points the state machien provides. Parent attr rename
needs a new intent chain to be built, not run multiple independent
intent chains for each modification.Quoted 3 lines
It cant be an attr replace because technically the names are
different.I disagree - we have all the pieces we need in the state machine
already, we just need to define separate attr names for the remove and insert steps in the attr intent.That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation,
but we actually implement it as a "replace entire attribute" operation.Without LARP, we do that overwrite in independent steps via an
intermediate INCOMPLETE state to allow two xattrs of the same name
to exist in the attr tree at the same time. IOWs, the attr value
overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always
have either the old or new xattr visible.With LARP, we can remove the original attr first, thereby avoiding
the need for two versions of the xattr to exist in the tree in
the
first place. However, we have to do these two operations as a pair
of linked independent operations. The intent chain provides the
linking, and requires us to log the name and the value of the attr
that we are overwriting in the intent. Hence we can always recover
the modification to completion no matter where in the operation
we
fail.When it comes to a parent attr rename operation, we are effectively
doing two linked operations - remove the old attr, set the new
attr
- on different attributes. Implementation wise, it is exactly the
same sequence as a "replace value" operation, except for the fact
that the new attr we add has a different name.Quoted 5 lines
Hence the only real difference between the existing "attr replace"
and the intent chain we need for "parent attr rename" is that we
have to log two attr names instead of one.Quoted 9 lines
To be clear, this would imply expanding
xfs_attri_log_formatto have
anotheralfi_new_name_lenfeild and another iovec for the attr intent
right? Does that cause issues to change the on disk log layout after
the original has merged? Or is that ok for things that are still
experimental? Thanks!I think we can get away with this quite easily without breaking the
existing experimental code.struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint32_t __pad; /* pad to 64 bit aligned */ uint64_t alfi_id; /* attri identifier */ uint64_t alfi_ino; /* the inode for thisattr
operation */
uint32_talfi_op_flags; /* marks the op as a set
or
remove */uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
We have a padding field in there that is currently all zeros. Let's
make that a count of the number of {name, value} tuples that are appended to the format. i.e.struct
xfs_attri_log_name{
uint32_talfi_op_flags; /* marks the op as a set
or
remove */uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint8_t alfi_attr_cnt; /* count of name/val pairs */ uint8_t __pad1; /* pad to 64 bitaligned */
uint16_t__pad2; /* pad to 64 bit aligned */
uint64_talfi_id; /* attri identifier */
uint64_talfi_ino; /* the inode for this
attr
operation */struct xfs_attri_log_name alfi_attr[]; /* attrs to operate on*/
};Basically, the size and shape of the structure has not changed, and
ifalfi_attr_cnt== 0 we just treat it as ifalfi_attr_cnt== 1 as
the backwards compat code for the existing code.And then we just have as many followup regions for name/val pairs as are defined by the
alfi_attr_cntand alfi_attr[] parts of the structure. Each attr can have a different operation performed on them, and they can have different filters applied so they can exist
in different namespaces, too.Quoted 6 lines
SO I don't think we need a new on-disk feature bit for this enhancement - it definitely comes under the heading of "this stuff
is experimental, this is the sort of early structure revision that
EXPERIMENTAL is supposed to cover....You might even callit "
alfi_extra_names" to avoid the "0 means 1" stuff.
;)
Quoted 8 lines
Quoted 1 line
--D
Oh, I just noticed these comments this morning when I sent out the new
attri/d patch. I'll add this changes to v2. Please let me know if there's anything else you'd like me to change from the v1. Thx!Allison
Ok, so I am part way through coding this up, and I'm getting this feeling like this is not going to work out very well due to the size checks for the log formats:
root@garnet:/home/achender/work_area/xfs-linux# git diff fs/xfs/libxfs/xfs_log_format.h fs/xfs/xfs_ondisk.h
(quoted diff) +0 -0
b/fs/xfs/libxfs/xfs_log_format.h
index f1ff52ebb982..5a4e700f32fc 100644
fs/xfs/libxfs/xfs_log_format.h +2 -0
@@ -922,6 +922,13 @@ struct xfs_icreate_log { XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE) +struct xfs_attri_log_name { + uint32_t alfi_op_flags; /* marks the op as a set or
remove */
+ uint32_t alfi_name_len; /* attr name length */ + uint32_t alfi_value_len; /* attr value length */ + uint32_t alfi_attr_filter;/* attr filter flags */
+};
+
/*
* This is the structure used to lay out an attr log item in the
* log.
@@ -929,14 +936,12 @@ struct xfs_icreate_log {
struct xfs_attri_log_format {
uint16_t alfi_type; /* attri log item type */
uint16_t alfi_size; /* size of this item */
- uint32_t __pad; /* pad to 64 bit aligned */
+ uint8_t alfi_extra_names;/* count of name/val pairs */
+ uint8_t __pad1; /* pad to 64 bit aligned */
+ uint16_t __pad2; /* pad to 64 bit aligned */
uint64_t alfi_id; /* attri identifier */
uint64_t alfi_ino; /* the inode for this attr
operation */
- uint32_t alfi_op_flags; /* marks the op as a set or
remove */
- uint32_t alfi_name_len; /* attr name length */
- uint32_t alfi_value_len; /* attr value length */
- uint32_t alfi_attr_filter;/* attr filter flags */
+ struct xfs_attri_log_name alfi_attr[]; /* attrs to operate on*/
};
struct xfs_attrd_log_format {
fs/xfs/xfs_ondisk.h +1 -1
@@ -132,7 +132,7 @@ xfs_check_ondisk_structs(void) XFS_CHECK_STRUCT_SIZE(struct xfs_inode_log_format, 56); XFS_CHECK_STRUCT_SIZE(struct xfs_qoff_logformat, 20); XFS_CHECK_STRUCT_SIZE(struct xfs_trans_header, 16); - XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 48); + XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 24); XFS_CHECK_STRUCT_SIZE(struct xfs_attrd_log_format, 16);/* parent pointer ioctls */
root@garnet:/home/achender/work_area/xfs-linux#
If the on disk size check thinks the format is 24 bytes, and then we surprise pack an array of structs after it, isnt that going to run over the next item? I think anything dynamic like this has to be an nvec.
Maybe we leave the existing alfi_* as they are so the size doesnt
change, and then if we have a value in alfi_extra_names, then we have an extra nvec that has the array in it. I think that would work.
FWIW, an alternate solution would be to use the pad for a second name length, and then we get a patch that's very similar to the one I sent out last Tues, but backward compatible. Though it does eat the remaining pad and wouldn't be as flexible, I cant think of an attr op that would need more than two names either?
Let me know what people think. Thanks!
Allison
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 264 lines · Alli
Quoted 215 lines · Alli
Quoted 214 lines · Darrick J. Wong
Quoted 202 lines · Dave Chinner
Quoted 126 lines · Alli
Quoted 119 lines · Dave Chinner
Quoted 31 lines · Alli
Quoted 24 lines · Dave Chinner
Quoted 12 lines · Darrick J. Wong
Quoted 9 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the
underlying
attr replay. A multi transaction replay currently performs a
single step of the replay, then deferrs the rest if there is
more
to do.Yup.
Quoted 6 lines
Quoted 6 lines
This causes race conditions with other attr replays that
might be recovered before the remaining deferred work has had
a
chance to finish.Quoted 6 lines
What other attr replays are we racing against? There can only be
one incomplete attr item intent/done chain per inode present in
log
recovery, right?Quoted 7 lines
No, a rename queues up a set and remove before committing the
transaction. One for the new parent pointer, and another to
remove
the
old one.Ah. That really needs to be described in the commit message - changing from "single intent chain per object" to "multiple concurrent independent and unserialised intent chains per object" is
a pretty important design rule change...The whole point of intents is to allow complex, multi-stage operations on a single object to be sequenced in a tightly controlled manner. They weren't intended to be run as concurrent
lines of modification on single items; if you need to do two modifications on an object, the intent chain ties the two modifications together into a single whole.One of the reasons I rewrote the attr state machine for LARP was to
enable new multiple attr operation chains to be easily build from
the entry points the state machien provides. Parent attr rename
needs a new intent chain to be built, not run multiple independent
intent chains for each modification.Quoted 3 lines
It cant be an attr replace because technically the names are
different.I disagree - we have all the pieces we need in the state machine
already, we just need to define separate attr names for the remove and insert steps in the attr intent.That is, the "replace" operation we execute when an attr set overwrites the value is "technically" a "replace value" operation,
but we actually implement it as a "replace entire attribute" operation.Without LARP, we do that overwrite in independent steps via an
intermediate INCOMPLETE state to allow two xattrs of the same name
to exist in the attr tree at the same time. IOWs, the attr value
overwrite is effectively a "set-swap-remove" operation on two entirely independent xattrs, ensuring that if we crash we always
have either the old or new xattr visible.With LARP, we can remove the original attr first, thereby avoiding
the need for two versions of the xattr to exist in the tree in
the
first place. However, we have to do these two operations as a pair
of linked independent operations. The intent chain provides the
linking, and requires us to log the name and the value of the attr
that we are overwriting in the intent. Hence we can always recover
the modification to completion no matter where in the operation
we
fail.When it comes to a parent attr rename operation, we are effectively
doing two linked operations - remove the old attr, set the new
attr
- on different attributes. Implementation wise, it is exactly the
same sequence as a "replace value" operation, except for the fact
that the new attr we add has a different name.Quoted 5 lines
Hence the only real difference between the existing "attr replace"
and the intent chain we need for "parent attr rename" is that we
have to log two attr names instead of one.Quoted 9 lines
To be clear, this would imply expanding
xfs_attri_log_formatto have
anotheralfi_new_name_lenfeild and another iovec for the attr intent
right? Does that cause issues to change the on disk log layout after
the original has merged? Or is that ok for things that are still
experimental? Thanks!I think we can get away with this quite easily without breaking the
existing experimental code.struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint32_t __pad; /* pad to 64 bit aligned */ uint64_t alfi_id; /* attri identifier */ uint64_t alfi_ino; /* the inode for thisattr
operation */
uint32_talfi_op_flags; /* marks the op as a set
or
remove */uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
We have a padding field in there that is currently all zeros. Let's
make that a count of the number of {name, value} tuples that are appended to the format. i.e.struct
xfs_attri_log_name{
uint32_talfi_op_flags; /* marks the op as a set
or
remove */uint32_t alfi_name_len; /* attr name length */ uint32_t alfi_value_len; /* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */};
struct
xfs_attri_log_format{uint16_t alfi_type; /* attri log item type */ uint16_t alfi_size; /* size of this item */ uint8_t alfi_attr_cnt; /* count of name/val pairs */ uint8_t __pad1; /* pad to 64 bitaligned */
uint16_t__pad2; /* pad to 64 bit aligned */
uint64_talfi_id; /* attri identifier */
uint64_talfi_ino; /* the inode for this
attr
operation */struct xfs_attri_log_name alfi_attr[]; /* attrs to operate on*/
};Basically, the size and shape of the structure has not changed, and
ifalfi_attr_cnt== 0 we just treat it as ifalfi_attr_cnt== 1 as
the backwards compat code for the existing code.And then we just have as many followup regions for name/val pairs as are defined by the
alfi_attr_cntand alfi_attr[] parts of the structure. Each attr can have a different operation performed on them, and they can have different filters applied so they can exist
in different namespaces, too.Quoted 6 lines
SO I don't think we need a new on-disk feature bit for this enhancement - it definitely comes under the heading of "this stuff
is experimental, this is the sort of early structure revision that
EXPERIMENTAL is supposed to cover....You might even callit "
alfi_extra_names" to avoid the "0 means 1" stuff.
;)Quoted 8 lines
Quoted 1 line
--D
Oh, I just noticed these comments this morning when I sent out the new
attri/d patch. I'll add this changes to v2. Please let me know if there's anything else you'd like me to change from the v1. Thx!Allison
Ok, so I am part way through coding this up, and I'm getting this feeling like this is not going to work out very well due to the size checks for the log formats:
root@garnet:/home/achender/work_area/xfs-linux# git diff fs/xfs/libxfs/xfs_log_format.h fs/xfs/xfs_ondisk.hfs/xfs/libxfs/xfs_log_format.h +2
-0@@ -922,6 +922,13 @@ struct xfs_icreate_log { XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE) +struct xfs_attri_log_name { + uint32_t alfi_op_flags; /* marks the op as a set orremove */
fs/xfs/libxfs/xfs_log_format.h +8
-1+uint32_t alfi_name_len;/* attr name length */ +uint32_t alfi_value_len;/* attr value length */ +uint32_t alfi_attr_filter;/* attr filter flags */ +}; +/* * This is the structure used to lay out an attr log item in the * log. @@ -929,14 +936,12 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t __pad;/* pad to 64 bit aligned */ +uint8_t alfi_extra_names;/* count of name/val pairs */ +uint8_t __pad1;/* pad to 64 bit aligned */ +uint16_t __pad2;/* pad to 64 bit aligned */ uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */
-uint32_talfi_op_flags; /* marks the op as a set or
remove */
fs/xfs/libxfs/xfs_log_format.h
Quoted 4 lines
fs/xfs/libxfs/xfs_log_format.h +1
-3-uint32_t alfi_name_len;/* attr name length */ -uint32_t alfi_value_len;/* attr value length */ -uint32_t alfi_attr_filter;/* attr filter flags */ +struct xfs_attri_log_name alfi_attr[];/* attrs to operate on
What's the length of this VLA? 1 for a normal SET or REPLACE operation, and 2 for the "rename and replace value" operation?
If so, why do we need two xfs_attri_log_name structures? The old value is unimportant, so we only need one alfi_value_len per operation. Each xfs_attri_log_format only describes one change, so it only needs one alfi_op_flags per op.
For now I also don't think attributes should be able to jump namespaces, so we'd only need one alfi_attr_filter per op as well.
*lightbulb comes on* Oops, I think I led you astray with my unfortunate comment. :(
IOWs, the only change to struct xfs_attri_log_format is:
- uint32_t __pad; /* pad to 64 bit aligned */ + uint32_t alfi_new_namelen;/* new attr name length */
and the rest of the changes in "[PATCH] xfs: Add new name to attri/d" are more or less fine as is.
I'll go reply to that before I get back to Dave's log accounting stuff.
--D
Quoted 45 lines
*/
};struct
xfs_attrd_log_format{fs/xfs/xfs_ondisk.h +1
-1@@ -132,7 +132,7 @@ xfs_check_ondisk_structs(void) XFS_CHECK_STRUCT_SIZE(struct xfs_inode_log_format, 56); XFS_CHECK_STRUCT_SIZE(struct xfs_qoff_logformat, 20); XFS_CHECK_STRUCT_SIZE(struct xfs_trans_header, 16); - XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 48); + XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 24); XFS_CHECK_STRUCT_SIZE(struct xfs_attrd_log_format, 16);/* parent pointer ioctls */ root@garnet:/home/achender/work_area/xfs-linux#
If the on disk size check thinks the format is 24 bytes, and then we surprise pack an array of structs after it, isnt that going to run over the next item? I think anything dynamic like this has to be an nvec.
Maybe we leave the existing alfi_* as they are so the size doesnt
change, and then if we have a value inalfi_extra_names, then we have an extra nvec that has the array in it. I think that would work.FWIW, an alternate solution would be to use the pad for a second name length, and then we get a patch that's very similar to the one I sent out last Tues, but backward compatible. Though it does eat the remaining pad and wouldn't be as flexible, I cant think of an attr op that would need more than two names either?
Let me know what people think. Thanks!
AllisonQuoted 6 lines
Quoted 6 lines
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
Quoted 360 lines · Darrick J. Wong
Quoted 334 lines · Alli
Quoted 279 lines · Alli
Quoted 271 lines · Darrick J. Wong
Quoted 263 lines · Dave Chinner
Quoted 159 lines · Alli
Quoted 151 lines · Dave Chinner
Quoted 38 lines · Alli
Quoted 30 lines · Dave Chinner
Quoted 15 lines · Darrick J. Wong
Quoted 4 lines · Allison Henderson
Recent parent pointer testing has exposed a bug
in
the
underlyingQuoted 8 lines
attr replay. A multi transaction replay
currently
performs a
single step of the replay, then deferrs the rest
if
there is
more
to do.Yup.
Quoted 8 lines
Quoted 8 lines
This causes race conditions with other attr
replays
that
might be recovered before the remaining deferred
work
has had
a
chance to finish.Quoted 7 lines
What other attr replays are we racing against? There can
only be
one incomplete attr item intent/done chain per inode present in
log
recovery, right?Quoted 9 lines
No, a rename queues up a set and remove before
committing
the
transaction. One for the new parent pointer, and
another
to
remove
the
old one.Ah. That really needs to be described in the commit message -
changing from "single intent chain per object" to "multiple
concurrent independent and unserialised intent chains per object" is
a pretty important design rule change...The whole point of intents is to allow complex, multi- stage
operations on a single object to be sequenced in a tightly
controlled manner. They weren't intended to be run as concurrent
lines of modification on single items; if you need to do two
modifications on an object, the intent chain ties the two modifications together into a single whole.One of the reasons I rewrote the attr state machine for LARP
was to
enable new multiple attr operation chains to be easily build
from
the entry points the state machien provides. Parent attr rename
needs a new intent chain to be built, not run multiple independent
intent chains for each modification.Quoted 4 lines
It cant be an attr replace because technically the names
are
different.I disagree - we have all the pieces we need in the state machine
already, we just need to define separate attr names for the
remove and insert steps in the attr intent.That is, the "replace" operation we execute when an attr set
overwrites the value is "technically" a "replace value" operation,
but we actually implement it as a "replace entire attribute"
operation.Without LARP, we do that overwrite in independent steps via
an
intermediate INCOMPLETE state to allow two xattrs of the same
name
to exist in the attr tree at the same time. IOWs, the attr
value
overwrite is effectively a "set-swap-remove" operation on two
entirely independent xattrs, ensuring that if we crash we always
have either the old or new xattr visible.With LARP, we can remove the original attr first, thereby avoiding
the need for two versions of the xattr to exist in the tree
in
the
first place. However, we have to do these two operations as a
pair
of linked independent operations. The intent chain provides
the
linking, and requires us to log the name and the value of the
attr
that we are overwriting in the intent. Hence we can always
recover
the modification to completion no matter where in the operation
we
fail.When it comes to a parent attr rename operation, we are effectively
doing two linked operations - remove the old attr, set the
new
attr
- on different attributes. Implementation wise, it is exactly
the
same sequence as a "replace value" operation, except for the
fact
that the new attr we add has a different name.Quoted 6 lines
Hence the only real difference between the existing "attr replace"
and the intent chain we need for "parent attr rename" is that
we
have to log two attr names instead of one.Quoted 12 lines
To be clear, this would imply expanding
xfs_attri_log_formatto
have
anotheralfi_new_name_lenfeild and another iovec for the attr
intent
right? Does that cause issues to change the on disk log layout
after
the original has merged? Or is that ok for things that are still
experimental? Thanks!I think we can get away with this quite easily without breaking
the
existing experimental code.struct
xfs_attri_log_format{
uint16_talfi_type; /* attri log item
type */
uint16_talfi_size; /* size of this item
*/
uint32_t__pad; /* pad to 64 bit
aligned
*/
uint64_talfi_id; /* attri identifier
*/
uint64_talfi_ino; /* the inode for this
attr
operation */
uint32_talfi_op_flags; /* marks the op as a
set
or
remove */
uint32_talfi_name_len; /* attr name length
*/
uint32_talfi_value_len; /* attr value length
*/
uint32_talfi_attr_filter;/* attr filter flags
*/
};We have a padding field in there that is currently all zeros. Let's
make that a count of the number of {name, value} tuples that are
appended to the format. i.e.struct
xfs_attri_log_name{
uint32_talfi_op_flags; /* marks the op as a
set
or
remove */
uint32_talfi_name_len; /* attr name length
*/
uint32_talfi_value_len; /* attr value length
*/
uint32_talfi_attr_filter;/* attr filter flags
*/
};struct
xfs_attri_log_format{
uint16_talfi_type; /* attri log item
type */
uint16_talfi_size; /* size of this item
*/
uint8_talfi_attr_cnt; /* count of name/val
pairs
*/uint8_t __pad1; /* pad to 64 bit aligned */ uint16_t __pad2; /* pad to 64 bitaligned */
uint64_talfi_id; /* attri identifier
*/
uint64_talfi_ino; /* the inode for this
attr
operation */struct xfs_attri_log_name alfi_attr[]; /* attrs tooperate on
*/
};Basically, the size and shape of the structure has not changed,
and
ifalfi_attr_cnt== 0 we just treat it as ifalfi_attr_cnt== 1
as
the backwards compat code for the existing code.And then we just have as many followup regions for name/val pairs
as are defined by thealfi_attr_cntand alfi_attr[] parts of the
structure. Each attr can have a different operation performed on
them, and they can have different filters applied so they can exist
in different namespaces, too.Quoted 6 lines
SO I don't think we need a new on-disk feature bit for this enhancement - it definitely comes under the heading of "this stuff
is experimental, this is the sort of early structure revision that
EXPERIMENTAL is supposed to cover....Quoted 6 lines
You might even callit "
alfi_extra_names" to avoid the "0 means 1"
stuff.
;)--D
Quoted 8 lines
Oh, I just noticed these comments this morning when I sent out the
new
attri/d patch. I'll add this changes to v2. Please let me know if
there's anything else you'd like me to change from the v1. Thx!Allison
Ok, so I am part way through coding this up, and I'm getting this feeling like this is not going to work out very well due to the size
checks for the log formats:root@garnet:/home/achender/work_area/xfs-linux# git diff fs/xfs/libxfs/xfs_log_format.h fs/xfs/xfs_ondisk.hfs/xfs/libxfs/xfs_log_format.h +2
-0@@ -922,6 +922,13 @@ struct xfs_icreate_log { XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE) +struct xfs_attri_log_name { + uint32_t alfi_op_flags; /* marks the op as a set orremove */
fs/xfs/libxfs/xfs_log_format.h +8
-1+uint32_t alfi_name_len;/* attr name length */ +uint32_t alfi_value_len;/* attr value length */ +uint32_t alfi_attr_filter;/* attr filter flags */ +}; +/* * This is the structure used to lay out an attr log item in the * log. @@ -929,14 +936,12 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t __pad;/* pad to 64 bit aligned */ +uint8_t alfi_extra_names;/* count of name/val pairs */ +uint8_t __pad1;/* pad to 64 bit aligned */ +uint16_t __pad2;/* pad to 64 bit aligned */ uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */
-uint32_talfi_op_flags; /* marks the op as a set or
remove */fs/xfs/libxfs/xfs_log_format.h +1
-3-uint32_t alfi_name_len;/* attr name length */ -uint32_t alfi_value_len;/* attr value length */ -uint32_t alfi_attr_filter;/* attr filter flags */ +struct xfs_attri_log_name alfi_attr[];/* attrs to operate Quoted 1 line
on
What's the length of this VLA? 1 for a normal SET or REPLACE operation, and 2 for the "rename and replace value" operation?
If so, why do we need two
xfs_attri_log_namestructures? The old value
is unimportant, so we only need onealfi_value_lenper operation. Eachxfs_attri_log_formatonly describes one change, so it only needs onealfi_op_flagsper op.For now I also don't think attributes should be able to jump namespaces,
so we'd only need onealfi_attr_filterper op as well.*lightbulb comes on* Oops, I think I led you astray with my unfortunate
comment. :(IOWs, the only change to struct
xfs_attri_log_formatis:(quoted diff) +1
-1- uint32_t __pad; /* pad to 64 bit aligned */ + uint32_t alfi_new_namelen;/* new attr name length */
Quoted 7 lines
and the rest of the changes in "[PATCH] xfs: Add new name to attri/d" are more or less fine as is.
I'll go reply to that before I get back to Dave's log accounting stuff.
--D
Alrighty, I think thats the simplest solution for now. Will switch to that thread....
Quoted 58 lines
Quoted 58 lines
*/
};struct
xfs_attrd_log_format{fs/xfs/xfs_ondisk.h +0
-1@@ -132,7 +132,7 @@ xfs_check_ondisk_structs(void)XFS_CHECK_STRUCT_SIZE (struct xfs_inode_log_format, 56);XFS_CHECK_STRUCT_SIZE (struct xfs_qoff_logformat, 20);XFS_CHECK_STRUCT_SIZE (struct xfs_trans_header, 16); - XFS_CHECK_STRUCT_SIZE(struct
xfs_attri_log_format, 48);
+ XFS_CHECK_STRUCT_SIZE(structxfs_attri_log_format, 24);
XFS_CHECK_STRUCT_SIZE(structxfs_attrd_log_format, 16);/* parent pointer ioctls */root@garnet:/home/achender/work_area/xfs-linux#
If the on disk size check thinks the format is 24 bytes, and then we
surprise pack an array of structs after it, isnt that going to run over
the next item? I think anything dynamic like this has to be an nvec.
Maybe we leave the existing alfi_* as they are so the size doesnt
change, and then if we have a value inalfi_extra_names, then we have
an extra nvec that has the array in it. I think that would work.FWIW, an alternate solution would be to use the pad for a second name
length, and then we get a patch that's very similar to the one I sent
out last Tues, but backward compatible. Though it does eat the remaining pad and wouldn't be as flexible, I cant think of an attr op
that would need more than two names either?Let me know what people think. Thanks!
AllisonQuoted 6 lines
Quoted 6 lines
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH RESEND v2 01/18] xfs: Fix multi-transaction larp replay
In reply to Darrick J. WongQuoted 96 lines · Darrick J. Wong
Quoted 63 lines · Allison Henderson
Recent parent pointer testing has exposed a bug in the underlying attr replay. A multi transaction replay currently performs a single step of the replay, then deferrs the rest if there is more to do. This causes race conditions with other attr replays that might be recovered before the remaining deferred work has had a chance to finish. This can lead to interleaved set and remove operations that may clobber the attribute fork. Fix this by deferring all work for any attribute operation.
Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_attr_item.c | 35 ++++++++--------------------------- 1 file changed, 8 insertions(+), 27 deletions(-)fs/xfs/xfs_attr_item.c +5
-8@@ -635,52 +635,33 @@ xfs_attri_item_recover(break ;case XFS_ATTRI_OP_FLAGS_REMOVE :if (!xfs_inode_hasattr (args->dp)) - goto out; +return 0 ; attr->xattri_dela_state =xfs_attr_init_remove_state (args);break ;default :ASSERT (0 ); - error = -EFSCORRUPTED ; - goto out; +return -EFSCORRUPTED ; }xfs_init_attr_trans (args, &tres, &total); error =xfs_trans_alloc (mp, &tres, total,0 ,XFS_TRANS_RESERVE , &tp);if (error) - goto out; +return error; args->trans = tp; done_item =xfs_trans_get_attrd (tp, attrip); + args->trans->t_flags |=XFS_TRANS_HAS_INTENT_DONE ; +set_bit (XFS_LI_DIRTY , &done_item->attrd_item.li_flags);xfs_ilock (ip,XFS_ILOCK_EXCL );xfs_trans_ijoin (tp, ip,0 ); - error =xfs_xattri_finish_update (attr, done_item); -if (error == -EAGAIN ) { -/* -* There's more work to do, so add the intent item to this
fs/xfs/xfs_attr_item.c +0
-3-* transaction so that we can continue it later. -*/ - xfs_defer_add(tp, XFS_DEFER_OPS_TYPE_ATTR, &attr-Quoted 1 line
xattri_list);fs/xfs/xfs_attr_item.c +0
-1- error = xfs_defer_ops_capture_and_commit(tp,
capture_list);fs/xfs/xfs_attr_item.c +0
-3-if (error) - goto out_unlock; -fs/xfs/xfs_attr_item.c xfs_attri_item_recover( · line 659
Quoted 10 lines
fs/xfs/xfs_attr_item.c +1
-9-xfs_iunlock (ip,XFS_ILOCK_EXCL ); -xfs_irele (ip); -return 0 ; - } -if (error) { -xfs_trans_cancel (tp); - goto out_unlock; - } - +xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_ATTR , &attr->xattri_list);This seems a little convoluted to me. Maybe? Maybe not?
1. Log recovery recreates an incore
xfs_attri_log_itemfrom what it finds in the log.2. This function then logs an xattrd for the recovered xattri item.
3. Then it creates a new
xfs_attr_intentto complete the operation.4. Finally, it calls
xfs_defer_ops_capture_and_commit, which logs a new
xattri for the intent created in step 3 and also commits the xattrd for
the first xattri.IOWs, the only difference between before and after is that we're not advancing one more step through the state machine as part of log recovery. From the perspective of the log, the recovery function merely
replaces the recovered xattri log item with a new one.
Quoted 5 lines
Why can't we just attach the recovered xattri to the
xfs_defer_pending
that is created to point to thexfs_attr_intentthat's created in step
3, and skip the xattrd?
Oh, I see. I hadnt thought of doing it that way, this was based on the initial solution suggested to the first patch of v1 (xfs: Add larp state XFS_DAS_CREATE_FORK). But what you mention below also makes sense. So I suppose if no one has any gripes then maybe it should stay as it is then. Thx for the reviews!
Allison
Quoted 30 lines
I /think/ the answer to that question is that we might need to move the
log tail forward to free enough log space to finish the intent items, so
creating the extra xattrd/xattri (a) avoid the complexity of submitting
an incore intent item *and* a log intent item to the defer ops machinery; and (b) avoid livelocks in log recovery. Therefore, we actually need to do it this way.IOWS, I *think* this is ok, but want to see if others have differing perspectives on how log item recovery works?
--D
Quoted 14 lines
fs/xfs/xfs_attr_item.c +2
-3error =xfs_defer_ops_capture_and_commit (tp, capture_list); -out_unlock: +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_irele (ip); -out: -xfs_attr_free_item (attr); +return error; }Signature · 2 lines
-- 2.25.1
PATCH RESEND v2 2/18 xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
In reply to Allison HendersonRenames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.
--- fs/xfs/libxfs/xfs_defer.c | 28 ++++++++++++++++++++++++++-- fs/xfs/libxfs/xfs_defer.h | 8 +++++++- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 1 + 4 files changed, 35 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_defer.c +26 -2
@@ -820,13 +820,37 @@ xfs_defer_ops_continue(struct xfs_trans *tp, struct xfs_defer_resources *dres) { - unsigned int i; + unsigned int i, j; +struct xfs_inode *sips[XFS_DEFER_OPS_NR_INODES ]; +struct xfs_inode *temp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );ASSERT (!(tp->t_flags &XFS_TRANS_DIRTY ));/* Lock the captured resources to the new transaction. */ -if (dfc->dfc_held.dr_inos ==2 ) +if (dfc->dfc_held.dr_inos >2 ) { +/* +* Renames with parent pointer updates can lock up to 5 inodes, +* sorted by their inode number. So we need to make sure they +* are relocked in the same way. +*/ +memset (sips,0 ,sizeof (sips)); +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) + sips[i] = dfc->dfc_held.dr_ip[i]; + +/* Bubble sort of at most 5 inodes */ +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) { +for (j =1 ; j < dfc->dfc_held.dr_inos; j++) { +if (sips[j]->i_ino < sips[j-1 ]->i_ino) { + temp = sips[j]; + sips[j] = sips[j-1 ]; + sips[j-1 ] = temp; + } + } + } + +xfs_lock_inodes (sips, dfc->dfc_held.dr_inos,XFS_ILOCK_EXCL ); + }else if (dfc->dfc_held.dr_inos ==2 )xfs_lock_two_inodes (dfc->dfc_held.dr_ip[0 ],XFS_ILOCK_EXCL , dfc->dfc_held.dr_ip[1 ],XFS_ILOCK_EXCL );else if (dfc->dfc_held.dr_inos ==1 )
fs/xfs/libxfs/xfs_defer.h +7 -1
@@ -70,7 +70,13 @@ extern const struct xfs_defer_op_type xfs_attr_defer_type;/* * Deferred operation item relogging limits. */ -#define XFS_DEFER_OPS_NR_INODES 2/* join up to two inodes */ + +/* +* Rename w/ parent pointers can require up to 5 inodes with defered ops to +* be joined to the transaction: src_dp, target_dp, src_ip, target_ip, and wip. +* These inodes are locked in sorted order by their inode numbers +*/ +#define XFS_DEFER_OPS_NR_INODES 5#define XFS_DEFER_OPS_NR_BUFS 2/* join up to two buffers */ /* Resources that must be held across a transaction roll. */
fs/xfs/xfs_inode.c +1 -1
@@ -447,7 +447,7 @@ xfs_lock_inumorder(* lock more than one at a time, lockdep will report false positives saying we * have violated locking orders. */ -static void +void xfs_lock_inodes( struct xfs_inode **ips, int inodes,
fs/xfs/xfs_inode.h +1 -0
@@ -573,5 +573,6 @@ void xfs_end_io(struct work_struct *work);int xfs_ilock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_iunlock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2); +void xfs_lock_inodes (struct xfs_inode **ips,int inodes,uint lock_mode);#endif /* __XFS_INODE_H__ */
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 02/18] xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
Quoted 40 lines · Allison Henderson
Renames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.
Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/libxfs/xfs_defer.c | 28 ++++++++++++++++++++++++++-- fs/xfs/libxfs/xfs_defer.h | 8 +++++++- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 1 + 4 files changed, 35 insertions(+), 4 deletions(-)fs/xfs/libxfs/xfs_defer.c +13
-2@@ -820,13 +820,37 @@ xfs_defer_ops_continue(struct xfs_trans *tp, struct xfs_defer_resources *dres) { - unsigned int i; + unsigned int i, j; +struct xfs_inode *sips[XFS_DEFER_OPS_NR_INODES ]; +struct xfs_inode *temp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );ASSERT (!(tp->t_flags &XFS_TRANS_DIRTY ));/* Lock the captured resources to the new transaction. */ -if (dfc->dfc_held.dr_inos ==2 ) +if (dfc->dfc_held.dr_inos >2 ) { +/* +* Renames with parent pointer updates can lock up to 5 inodes, +* sorted by their inode number. So we need to make sure they +* are relocked in the same way. +*/ +memset (sips,0 ,sizeof (sips)); +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) + sips[i] = dfc->dfc_held.dr_ip[i]; +
fs/xfs/libxfs/xfs_defer.c xfs_defer_ops_continue( · line 850
Quoted 10 lines
fs/xfs/libxfs/xfs_defer.c +10
-0+/* Bubble sort of at most 5 inodes */ +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) { +for (j =1 ; j < dfc->dfc_held.dr_inos; j++) { +if (sips[j]->i_ino < sips[j-1 ]->i_ino) { + temp = sips[j]; + sips[j] = sips[j-1 ]; + sips[j-1 ] = temp; + } + } + }
Why not reuse xfs_sort_for_rename?
I also wonder if it's worth the trouble to replace the open-coded bubblesort with a call to sort_r(), but TBH I suspect the cost of a retpoline for the compare function isn't worth the overhead.
Quoted 11 lines
fs/xfs/libxfs/xfs_defer.c +3
-0+ +xfs_lock_inodes (sips, dfc->dfc_held.dr_inos,XFS_ILOCK_EXCL ); + }else if (dfc->dfc_held.dr_inos ==2 )xfs_lock_two_inodes (dfc->dfc_held.dr_ip[0 ],XFS_ILOCK_EXCL , dfc->dfc_held.dr_ip[1 ],XFS_ILOCK_EXCL );else if (dfc->dfc_held.dr_inos ==1 )fs/xfs/libxfs/xfs_defer.h +0
-0
fs/xfs/libxfs/xfs_defer.h extern const struct xfs_defer_op_type xfs_attr_defer_type; · line 77
Quoted 9 lines
fs/xfs/libxfs/xfs_defer.h +5
-1@@ -70,7 +70,13 @@ extern const struct xfs_defer_op_type xfs_attr_defer_type;/* * Deferred operation item relogging limits. */ -#define XFS_DEFER_OPS_NR_INODES 2/* join up to two inodes */ + +/* +* Rename w/ parent pointers can require up to 5 inodes with defered ops to +* be joined to the transaction: src_dp, target_dp, src_ip, target_ip, and wip. +* These inodes are locked in sorted order by their inode numbers
Much inode. Thanks for recording this.
--D
Quoted 32 lines
fs/xfs/libxfs/xfs_defer.h +2
-0+*/ +#define XFS_DEFER_OPS_NR_INODES 5#define XFS_DEFER_OPS_NR_BUFS 2/* join up to two buffers */ /* Resources that must be held across a transaction roll. */ fs/xfs/xfs_inode.c +1
-1@@ -447,7 +447,7 @@ xfs_lock_inumorder(* lock more than one at a time, lockdep will report false positives saying we * have violated locking orders. */ -static void +void xfs_lock_inodes( struct xfs_inode **ips, int inodes,fs/xfs/xfs_inode.h +1
-0@@ -573,5 +573,6 @@ void xfs_end_io(struct work_struct *work);int xfs_ilock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_iunlock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2); +void xfs_lock_inodes (struct xfs_inode **ips,int inodes,uint lock_mode);#endif /* __XFS_INODE_H__ */ Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 02/18] xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
Quoted 48 lines · Darrick J. Wong
Quoted 47 lines · Allison Henderson
Renames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.
Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/libxfs/xfs_defer.c | 28 ++++++++++++++++++++++++++-- fs/xfs/libxfs/xfs_defer.h | 8 +++++++- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 1 + 4 files changed, 35 insertions(+), 4 deletions(-)fs/xfs/libxfs/xfs_defer.c +7
-2@@ -820,13 +820,37 @@ xfs_defer_ops_continue(struct xfs_trans *tp, struct xfs_defer_resources *dres) { - unsigned int i; + unsigned int i, j; +struct xfs_inode *sips[XFS_DEFER_OPS_NR_INODES ]; +struct xfs_inode *temp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );ASSERT (!(tp->t_flags &XFS_TRANS_DIRTY ));/* Lock the captured resources to the new transaction. */ -if (dfc->dfc_held.dr_inos ==2 ) +if (dfc->dfc_held.dr_inos >2 ) { +/* +* Renames with parent pointer updates can lock up to 5 inodes, +* sorted by their inode number. So we need to make sure they
fs/xfs/libxfs/xfs_defer.c +10
-0+* are relocked in the same way. +*/ +memset (sips,0 ,sizeof (sips)); +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) + sips[i] = dfc->dfc_held.dr_ip[i]; + +/* Bubble sort of at most 5 inodes */ +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) { +for (j =1 ; j < dfc->dfc_held.dr_inos; j++) { +if (sips[j]->i_ino < sips[j-1 ]->i_ino){
Quoted 8 lines
fs/xfs/libxfs/xfs_defer.c xfs_defer_ops_continue( · line 850
Quoted 6 lines
fs/xfs/libxfs/xfs_defer.c +6
-0+ temp = sips[j]; + sips[j] = sips[j-1 ]; + sips[j-1 ] = temp; + } + } + }Why not reuse
xfs_sort_for_rename?
Initially I had looked at doing that, but it would need some refactoring as it is not meant for an arbitrary number of inodes. Either some logic specific to rename would get pulled up, or we'd need another helper to repackage the parameters, but it's such a small bit of code, I'm not sure it saves much LOC either way.
Quoted 4 lines
I also wonder if it's worth the trouble to replace the open-coded bubblesort with a call to
sort_r(), but TBH I suspect the cost of a retpoline for the compare function isn't worth the overhead.
Yeah, it would make sense if there was lot of other places we sorted inodes, but with only two callers it does seem like a bit much.
I am fine with what ever method folks prefer tho.
Quoted 21 lines
Quoted 21 lines
fs/xfs/libxfs/xfs_defer.c +2
-0+ + xfs_lock_inodes(sips, dfc->dfc_held.dr_inos,
XFS_ILOCK_EXCL);fs/xfs/libxfs/xfs_defer.c +1
-0+ }else if (dfc->dfc_held.dr_inos ==2 ) xfs_lock_two_inodes(dfc->dfc_held.dr_ip[0],
XFS_ILOCK_EXCL,
dfc->dfc_held.dr_ip[1],XFS_ILOCK_EXCL);fs/xfs/libxfs/xfs_defer.c +0
-0else if (dfc->dfc_held.dr_inos ==1 )fs/xfs/libxfs/xfs_defer.h +1
-1@@ -70,7 +70,13 @@ extern const struct xfs_defer_op_type xfs_attr_defer_type;/* * Deferred operation item relogging limits. */ -#define XFS_DEFER_OPS_NR_INODES 2/* join up to two inodes */ +
Quoted 8 lines
Quoted 6 lines
fs/xfs/libxfs/xfs_defer.h +2
-0+/* +* Rename w/ parent pointers can require up to 5 inodes with defered ops to
+ * be joined to the transaction:src_dp,target_dp,src_ip,target_ip, and wip.
+ * These inodes are locked in sorted order by their inode numbersMuch inode. Thanks for recording this.
Sure, thx for the reviews!
Allison
Quoted 40 lines
--D
Quoted 37 lines
+ */
+#defineXFS_DEFER_OPS_NR_INODES5
#defineXFS_DEFER_OPS_NR_BUFS2 /* join up to two buffers
*//* Resources that must be held across a transaction roll. */
fs/xfs/xfs_inode.c +0
-0@@ -447,7 +447,7 @@ xfs_lock_inumorder(* lock more than one at a time, lockdep will report false positives saying we
* have violated locking orders.
*/fs/xfs/xfs_inode.c +1
-1-static void +void xfs_lock_inodes( struct xfs_inode **ips, int inodes,fs/xfs/xfs_inode.h +1
-0@@ -573,5 +573,6 @@ void xfs_end_io(struct work_struct *work);int xfs_ilock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_iunlock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2); +void xfs_lock_inodes(struct xfs_inode **ips, int inodes, uint
lock_mode);#endif /*
__XFS_INODE_H__*/Signature · 1 line
2.25.1
PATCH RESEND v2 3/18 xfs: Hold inode locks in xfs_ialloc
In reply to Allison HendersonModify xfs_ialloc to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 6 +++++- fs/xfs/xfs_qm.c | 4 +++- fs/xfs/xfs_symlink.c | 3 +++ 3 files changed, 11 insertions(+), 2 deletions(-)
fs/xfs/xfs_inode.c +5 -1
@@ -774,6 +774,8 @@ xfs_inode_inherit_flags2(/* * Initialise a newly allocated inode and return the in-core inode to the * caller locked exclusively. +* +* Caller is responsible for unlocking the inode manually upon return */ int xfs_init_new_inode(
@@ -900,7 +902,7 @@ xfs_init_new_inode(/* * Log the new values stuffed into the inode. */ -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, ip,0 );xfs_trans_log_inode (tp, ip, flags);/* now that we have an i_mode we can setup the inode structure */
@@ -1077,6 +1079,7 @@ xfs_create(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -1173,6 +1176,7 @@ xfs_create_tmpfile(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
fs/xfs/xfs_qm.c +3 -1
@@ -817,8 +817,10 @@ xfs_qm_qino_alloc(ASSERT (xfs_is_shutdown (mp));xfs_alert (mp,"%s failed (error %d)!" , __func__, error); } -if (need_alloc) +if (need_alloc) {xfs_finish_inode_setup (*ipp); +xfs_iunlock (*ipp,XFS_ILOCK_EXCL ); + }return error; }
fs/xfs/xfs_symlink.c +3 -0
@@ -337,6 +337,7 @@ xfs_symlink(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -358,6 +359,8 @@ xfs_symlink(if (unlock_dp_on_error)xfs_iunlock (dp,XFS_ILOCK_EXCL ); +if (ip) +xfs_iunlock (ip,XFS_ILOCK_EXCL );return error; }
Signature · 1 line
2.25.1
PATCH RESEND v2 4/18 xfs: Hold inode locks in xfs_trans_alloc_dir
In reply to Allison HendersonModify xfs_trans_alloc_dir to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 14 ++++++++++++-- fs/xfs/xfs_trans.c | 6 ++++-- 2 files changed, 16 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +12 -2
@@ -1278,10 +1278,15 @@ xfs_link(if (xfs_has_wsync (mp) ||xfs_has_dirsync (mp))xfs_trans_set_sync (tp); -return xfs_trans_commit (tp); + error =xfs_trans_commit (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); +return error; error_return:xfs_trans_cancel (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;
@@ -2517,15 +2522,20 @@ xfs_remove( error =xfs_trans_commit (tp);if (error) - goto std_return; + goto out_unlock;if (is_dir &&xfs_inode_is_filestream (ip))xfs_filestream_deassociate (ip); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:xfs_trans_cancel (tp); + out_unlock: +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL ); std_return:return error; }
fs/xfs/xfs_trans.c +4 -2
@@ -1356,6 +1356,8 @@ xfs_trans_alloc_ichange(* The caller must ensure that the on-disk dquots attached to this inode have * already been allocated and initialized. The ILOCKs will be dropped when the * transaction is committed or cancelled. +* +* Caller is responsible for unlocking the inodes manually upon return */ int xfs_trans_alloc_dir(
@@ -1386,8 +1388,8 @@ xfs_trans_alloc_dir(xfs_lock_two_inodes (dp,XFS_ILOCK_EXCL , ip,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); +xfs_trans_ijoin (tp, ip,0 ); error =xfs_qm_dqattach_locked (dp, false);if (error) {
Signature · 1 line
2.25.1
PATCH RESEND v2 5/18 xfs: get directory offset when adding directory name
In reply to Allison HendersonReturn the directory offset information when adding an entry to the directory.
This offset will be used as the parent pointer offset in xfs_create, xfs_symlink, xfs_link and xfs_rename.
[dchinner: forward ported and cleaned up] [dchinner: no s-o-b from Mark]
[bfoster: rebased, use args->geo in dir code] [achender: rebased, chaged __uint32_t to xfs_dir2_dataptr_t]
--- fs/xfs/libxfs/xfs_da_btree.h | 1 + fs/xfs/libxfs/xfs_dir2.c | 9 +++++++-- fs/xfs/libxfs/xfs_dir2.h | 2 +- fs/xfs/libxfs/xfs_dir2_block.c | 1 + fs/xfs/libxfs/xfs_dir2_leaf.c | 2 ++ fs/xfs/libxfs/xfs_dir2_node.c | 2 ++ fs/xfs/libxfs/xfs_dir2_sf.c | 2 ++ fs/xfs/xfs_inode.c | 6 +++--- fs/xfs/xfs_symlink.c | 3 ++- 9 files changed, 21 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_da_btree.h +1 -0
@@ -79,6 +79,7 @@ typedef struct xfs_da_args {int rmtvaluelen2;/* remote attr value length in bytes */ uint32_t op_flags;/* operation flags */ enum xfs_dacmp cmpresult;/* name compare result for lookups */ +xfs_dir2_dataptr_t offset;/* OUT: offset in directory */ } xfs_da_args_t;/*
fs/xfs/libxfs/xfs_dir2.c +7 -2
@@ -257,7 +257,8 @@ xfs_dir_createname(struct xfs_inode *dp, const struct xfs_name *name, xfs_ino_t inum, /* new entry inode number */ - xfs_extlen_t total) /* bmap's total block count */ + xfs_extlen_t total, /* bmap's total block count */ + xfs_dir2_dataptr_t *offset) /* OUT entry's dir offset */ { struct xfs_da_args *args;int rval;
@@ -312,6 +313,10 @@ xfs_dir_createname( rval =xfs_dir2_node_addname (args); out_free: +/* return the location that this entry was place in the parent inode */ +if (offset) + *offset = args->offset; +kmem_free (args);return rval; }
@@ -550,7 +555,7 @@ xfs_dir_canenter(xfs_inode_t *dp, struct xfs_name *name) /* name of entry to add */ { - return xfs_dir_createname(tp, dp, name, 0, 0); + return xfs_dir_createname(tp, dp, name, 0, 0, NULL); }/*
fs/xfs/libxfs/xfs_dir2.h +1 -1
@@ -40,7 +40,7 @@ extern int xfs_dir_init(struct xfs_trans *tp, struct xfs_inode *dp, struct xfs_inode *pdp);extern int xfs_dir_createname (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t inum, -xfs_extlen_t tot); +xfs_extlen_t tot,xfs_dir2_dataptr_t *offset);extern int xfs_dir_lookup (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t *inum,struct xfs_name *ci_name);
fs/xfs/libxfs/xfs_dir2_block.c +1 -0
@@ -573,6 +573,7 @@ xfs_dir2_block_addname(xfs_dir2_data_put_ftype (dp->i_mount, dep, args->filetype); tagp =xfs_dir2_data_entry_tag_p (dp->i_mount, dep); *tagp =cpu_to_be16 ((char *)dep - (char *)hdr); + args->offset =xfs_dir2_byte_to_dataptr ((char *)dep - (char *)hdr);/* * Clean up the bestfree array and log the header, tail, and entry. */
fs/xfs/libxfs/xfs_dir2_leaf.c +2 -0
@@ -865,6 +865,8 @@ xfs_dir2_leaf_addname(xfs_dir2_data_put_ftype (dp->i_mount, dep, args->filetype); tagp =xfs_dir2_data_entry_tag_p (dp->i_mount, dep); *tagp =cpu_to_be16 ((char *)dep - (char *)hdr); + args->offset =xfs_dir2_db_off_to_dataptr (args->geo, use_block, + (char *)dep - (char *)hdr);/* * Need to scan fix up the bestfree table. */
fs/xfs/libxfs/xfs_dir2_node.c +2 -0
@@ -1974,6 +1974,8 @@ xfs_dir2_node_addname_int(xfs_dir2_data_put_ftype (dp->i_mount, dep, args->filetype); tagp =xfs_dir2_data_entry_tag_p (dp->i_mount, dep); *tagp =cpu_to_be16 ((char *)dep - (char *)hdr); + args->offset =xfs_dir2_db_off_to_dataptr (args->geo, dbno, + (char *)dep - (char *)hdr);xfs_dir2_data_log_entry (args, dbp, dep);/* Rescan the freespace and log the data block if needed. */
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
@@ -485,6 +485,7 @@ xfs_dir2_sf_addname_easy(memcpy (sfep->name, args->name, sfep->namelen);xfs_dir2_sf_put_ino (mp, sfp, sfep, args->inumber);xfs_dir2_sf_put_ftype (mp, sfep, args->filetype); + args->offset =xfs_dir2_byte_to_dataptr (offset);/* * Update the header and inode.
@@ -575,6 +576,7 @@ xfs_dir2_sf_addname_hard(memcpy (sfep->name, args->name, sfep->namelen);xfs_dir2_sf_put_ino (mp, sfp, sfep, args->inumber);xfs_dir2_sf_put_ftype (mp, sfep, args->filetype); + args->offset =xfs_dir2_byte_to_dataptr (offset); sfp->count++;if (args->inumber >XFS_DIR2_MAX_SHORT_INUM && !objchange) sfp->i8count++;
fs/xfs/xfs_inode.c +3 -3
@@ -1039,7 +1039,7 @@ xfs_create( unlock_dp_on_error = false; error =xfs_dir_createname (tp, dp, name, ip->i_ino, - resblks -XFS_IALLOC_SPACE_RES (mp)); + resblks -XFS_IALLOC_SPACE_RES (mp),NULL );if (error) {ASSERT (error != -ENOSPC ); goto out_trans_cancel;
@@ -1262,7 +1262,7 @@ xfs_link( } error =xfs_dir_createname (tp, tdp, target_name, sip->i_ino, - resblks); + resblks,NULL );if (error) goto error_return;xfs_trans_ichgtime (tp, tdp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );
@@ -2983,7 +2983,7 @@ xfs_rename(* to account for the ".." reference from the new entry. */ error =xfs_dir_createname (tp, target_dp, target_name, - src_ip->i_ino, spaceres); + src_ip->i_ino, spaceres,NULL );if (error) goto out_trans_cancel;
fs/xfs/xfs_symlink.c +2 -1
@@ -314,7 +314,8 @@ xfs_symlink(/* * Create the directory entry for the symlink. */ - error =xfs_dir_createname (tp, dp, link_name, ip->i_ino, resblks); + error =xfs_dir_createname (tp, dp, link_name, + ip->i_ino, resblks,NULL );if (error) goto out_trans_cancel;xfs_trans_ichgtime (tp, dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );
Signature · 1 line
2.25.1
PATCH RESEND v2 6/18 xfs: get directory offset when removing directory name
In reply to Allison HendersonReturn the directory offset information when removing an entry to the directory.
This offset will be used as the parent pointer offset in xfs_remove.
[dchinner: forward ported and cleaned up]
[achender: rebased, changed __unint32_t to xfs_dir2_dataptr_t,
Changed typedefs to raw struct types]
--- fs/xfs/libxfs/xfs_dir2.c | 6 +++++- fs/xfs/libxfs/xfs_dir2.h | 3 ++- fs/xfs/libxfs/xfs_dir2_block.c | 4 ++-- fs/xfs/libxfs/xfs_dir2_leaf.c | 5 +++-- fs/xfs/libxfs/xfs_dir2_node.c | 5 +++-- fs/xfs/libxfs/xfs_dir2_sf.c | 2 ++ fs/xfs/xfs_inode.c | 4 ++-- 7 files changed, 19 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +5 -1
@@ -436,7 +436,8 @@ xfs_dir_removename(struct xfs_inode *dp, struct xfs_name *name, xfs_ino_t ino, - xfs_extlen_t total) /* bmap's total block count */ + xfs_extlen_t total, /* bmap's total block count */ + xfs_dir2_dataptr_t *offset) /* OUT: offset in directory */ { struct xfs_da_args *args;int rval;
@@ -481,6 +482,9 @@ xfs_dir_removename(else rval =xfs_dir2_node_removename (args); out_free: +if (offset) + *offset = args->offset; +kmem_free (args);return rval; }
fs/xfs/libxfs/xfs_dir2.h +2 -1
@@ -46,7 +46,8 @@ extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp, struct xfs_name *ci_name);extern int xfs_dir_removename (struct xfs_trans *tp,struct xfs_inode *dp,struct xfs_name *name,xfs_ino_t ino, -xfs_extlen_t tot); +xfs_extlen_t tot, +xfs_dir2_dataptr_t *offset);extern int xfs_dir_replace (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t inum,xfs_extlen_t tot);
fs/xfs/libxfs/xfs_dir2_block.c +2 -2
@@ -810,9 +810,9 @@ xfs_dir2_block_removename(/* * Point to the data entry using the leaf entry. */ + args->offset =be32_to_cpu (blp[ent].address); dep = (xfs_dir2_data_entry_t *)((char *)hdr + -xfs_dir2_dataptr_to_off (args->geo, -be32_to_cpu (blp[ent].address))); +xfs_dir2_dataptr_to_off (args->geo, args->offset));/* * Mark the data entry's space free. */
fs/xfs/libxfs/xfs_dir2_leaf.c +3 -2
@@ -1381,9 +1381,10 @@ xfs_dir2_leaf_removename(* Point to the leaf entry, use that to point to the data entry. */ lep = &leafhdr.ents[index]; - db =xfs_dir2_dataptr_to_db (geo,be32_to_cpu (lep->address)); + args->offset =be32_to_cpu (lep->address); + db =xfs_dir2_dataptr_to_db (args->geo, args->offset); dep = (xfs_dir2_data_entry_t *)((char *)hdr + -xfs_dir2_dataptr_to_off (geo,be32_to_cpu (lep->address))); +xfs_dir2_dataptr_to_off (args->geo, args->offset)); needscan = needlog =0 ; oldbest =be16_to_cpu (bf[0 ].length); ltp =xfs_dir2_leaf_tail_p (geo, leaf);
fs/xfs/libxfs/xfs_dir2_node.c +3 -2
@@ -1296,9 +1296,10 @@ xfs_dir2_leafn_remove(/* * Extract the data block and offset from the entry. */ - db =xfs_dir2_dataptr_to_db (geo,be32_to_cpu (lep->address)); + args->offset =be32_to_cpu (lep->address); + db =xfs_dir2_dataptr_to_db (args->geo, args->offset);ASSERT (dblk->blkno == db); - off =xfs_dir2_dataptr_to_off (geo,be32_to_cpu (lep->address)); + off =xfs_dir2_dataptr_to_off (args->geo, args->offset);ASSERT (dblk->index == off);/*
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
@@ -971,6 +971,8 @@ xfs_dir2_sf_removename( XFS_CMP_EXACT) {ASSERT (xfs_dir2_sf_get_ino (mp, sfp, sfep) == args->inumber); + args->offset =xfs_dir2_byte_to_dataptr ( +xfs_dir2_sf_get_offset (sfep));break ; } }
fs/xfs/xfs_inode.c +2 -2
@@ -2506,7 +2506,7 @@ xfs_remove(if (error) goto out_trans_cancel; - error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks); + error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks,NULL );if (error) {ASSERT (error != -ENOENT ); goto out_trans_cancel;
@@ -3080,7 +3080,7 @@ xfs_rename( spaceres); else error =xfs_dir_removename (tp, src_dp, src_name, src_ip->i_ino, - spaceres); + spaceres,NULL );if (error) goto out_trans_cancel;
Signature · 1 line
2.25.1
PATCH RESEND v2 7/18 xfs: get directory offset when replacing a directory name
In reply to Allison HendersonReturn the directory offset information when replacing an entry to the directory.
This offset will be used as the parent pointer offset in xfs_rename.
[dchinner: forward ported and cleaned up]
[achender: rebased, changed __unint32_t to xfs_dir2_dataptr_t,
Changed typedefs to raw struct types]
--- fs/xfs/libxfs/xfs_dir2.c | 8 ++++++-- fs/xfs/libxfs/xfs_dir2.h | 2 +- fs/xfs/libxfs/xfs_dir2_block.c | 4 ++-- fs/xfs/libxfs/xfs_dir2_leaf.c | 1 + fs/xfs/libxfs/xfs_dir2_node.c | 1 + fs/xfs/libxfs/xfs_dir2_sf.c | 2 ++ fs/xfs/xfs_inode.c | 16 ++++++++-------- 7 files changed, 21 insertions(+), 13 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +6 -2
@@ -482,7 +482,7 @@ xfs_dir_removename(else rval =xfs_dir2_node_removename (args); out_free: -if (offset) +if (!rval && offset) *offset = args->offset;kmem_free (args);
@@ -498,7 +498,8 @@ xfs_dir_replace(struct xfs_inode *dp, const struct xfs_name *name, /* name of entry to replace */ xfs_ino_t inum, /* new inode number */ - xfs_extlen_t total) /* bmap's total block count */ + xfs_extlen_t total, /* bmap's total block count */ + xfs_dir2_dataptr_t *offset) /* OUT: offset in directory */ { struct xfs_da_args *args;int rval;
@@ -546,6 +547,9 @@ xfs_dir_replace(else rval =xfs_dir2_node_replace (args); out_free: +if (offset) + *offset = args->offset; +kmem_free (args);return rval; }
fs/xfs/libxfs/xfs_dir2.h +1 -1
@@ -50,7 +50,7 @@ extern int xfs_dir_removename(struct xfs_trans *tp, struct xfs_inode *dp, xfs_dir2_dataptr_t *offset);extern int xfs_dir_replace (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t inum, -xfs_extlen_t tot); +xfs_extlen_t tot,xfs_dir2_dataptr_t *offset);extern int xfs_dir_canenter (struct xfs_trans *tp,struct xfs_inode *dp,struct xfs_name *name);
fs/xfs/libxfs/xfs_dir2_block.c +2 -2
@@ -885,9 +885,9 @@ xfs_dir2_block_replace(/* * Point to the data entry we need to change. */ + args->offset =be32_to_cpu (blp[ent].address); dep = (xfs_dir2_data_entry_t *)((char *)hdr + -xfs_dir2_dataptr_to_off (args->geo, -be32_to_cpu (blp[ent].address))); +xfs_dir2_dataptr_to_off (args->geo, args->offset));ASSERT (be64_to_cpu (dep->inumber) != args->inumber);/* * Change the inode number to the new value.
fs/xfs/libxfs/xfs_dir2_leaf.c +1 -0
@@ -1518,6 +1518,7 @@ xfs_dir2_leaf_replace(/* * Point to the data entry. */ + args->offset =be32_to_cpu (lep->address); dep = (xfs_dir2_data_entry_t *) ((char *)dbp->b_addr +xfs_dir2_dataptr_to_off (args->geo,be32_to_cpu (lep->address)));
fs/xfs/libxfs/xfs_dir2_node.c +1 -0
@@ -2242,6 +2242,7 @@ xfs_dir2_node_replace( hdr = state->extrablk.bp->b_addr;ASSERT (hdr->magic ==cpu_to_be32 (XFS_DIR2_DATA_MAGIC ) || hdr->magic ==cpu_to_be32 (XFS_DIR3_DATA_MAGIC )); + args->offset =be32_to_cpu (leafhdr.ents[blk->index].address); dep = (xfs_dir2_data_entry_t *) ((char *)hdr + xfs_dir2_dataptr_to_off(args->geo,
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
@@ -1109,6 +1109,8 @@ xfs_dir2_sf_replace(xfs_dir2_sf_put_ino (mp, sfp, sfep, args->inumber);xfs_dir2_sf_put_ftype (mp, sfep, args->filetype); + args->offset =xfs_dir2_byte_to_dataptr ( +xfs_dir2_sf_get_offset (sfep));break ; } }
fs/xfs/xfs_inode.c +8 -8
@@ -2487,7 +2487,7 @@ xfs_remove(*/ if (dp->i_ino != tp->t_mountp->m_sb.sb_rootino) { error =xfs_dir_replace (tp, ip, &xfs_name_dotdot, - tp->t_mountp->m_sb.sb_rootino,0 ); + tp->t_mountp->m_sb.sb_rootino,0 ,NULL );if (error)return error; }
@@ -2627,12 +2627,12 @@ xfs_cross_rename(int dp2_flags =0 ;/* Swap inode number for dirent in first parent */ - error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres); + error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres,NULL );if (error) goto out_trans_abort;/* Swap inode number for dirent in second parent */ - error =xfs_dir_replace (tp, dp2, name2, ip1->i_ino, spaceres); + error =xfs_dir_replace (tp, dp2, name2, ip1->i_ino, spaceres,NULL );if (error) goto out_trans_abort;
@@ -2646,7 +2646,7 @@ xfs_cross_rename(if (S_ISDIR (VFS_I (ip2)->i_mode)) { error =xfs_dir_replace (tp, ip2, &xfs_name_dotdot, - dp1->i_ino, spaceres); + dp1->i_ino, spaceres,NULL );if (error) goto out_trans_abort;
@@ -2670,7 +2670,7 @@ xfs_cross_rename(if (S_ISDIR (VFS_I (ip1)->i_mode)) { error =xfs_dir_replace (tp, ip1, &xfs_name_dotdot, - dp2->i_ino, spaceres); + dp2->i_ino, spaceres,NULL );if (error) goto out_trans_abort;
@@ -3004,7 +3004,7 @@ xfs_rename(* name at the destination directory, remove it first. */ error =xfs_dir_replace (tp, target_dp, target_name, - src_ip->i_ino, spaceres); + src_ip->i_ino, spaceres,NULL );if (error) goto out_trans_cancel;
@@ -3038,7 +3038,7 @@ xfs_rename(* directory. */ error =xfs_dir_replace (tp, src_ip, &xfs_name_dotdot, - target_dp->i_ino, spaceres); + target_dp->i_ino, spaceres,NULL );ASSERT (error != -EEXIST );if (error) goto out_trans_cancel;
@@ -3077,7 +3077,7 @@ xfs_rename(*/ if (wip) error =xfs_dir_replace (tp, src_dp, src_name, wip->i_ino, - spaceres); + spaceres,NULL );else error =xfs_dir_removename (tp, src_dp, src_name, src_ip->i_ino, spaceres,NULL );
Signature · 1 line
2.25.1
PATCH RESEND v2 8/18 xfs: add parent pointer support to attribute code
In reply to Allison HendersonAdd the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer entries; it uses reserved blocks like XFS_ATTR_ROOT.
[dchinner: forward ported and cleaned up]
[achender: rebased]
--- fs/xfs/libxfs/xfs_attr.c | 4 +++- fs/xfs/libxfs/xfs_da_format.h | 5 ++++- fs/xfs/libxfs/xfs_log_format.h | 1 + 3 files changed, 8 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr.c +3 -1
@@ -966,11 +966,13 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; -bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool rsvd;int error, local;int rmt_blks =0 ;unsigned int total; + rsvd = (args->attr_filter & (XFS_ATTR_ROOT |XFS_ATTR_PARENT )) !=0 ; +if (xfs_is_shutdown (dp->i_mount))return -EIO ;
fs/xfs/libxfs/xfs_da_format.h +4 -1
@@ -688,12 +688,15 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */ #define XFS_ATTR_SECURE_BIT 2/* limit access to secure attrs */ +#define XFS_ATTR_PARENT_BIT 3/* parent pointer attrs */ #define XFS_ATTR_INCOMPLETE_BIT 7/* attr in middle of create/delete */ #define XFS_ATTR_LOCAL (1u << XFS_ATTR_LOCAL_BIT)#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT) +#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_NSP_ONDISK_MASK \ + (XFS_ATTR_ROOT | XFS_ATTR_SECURE | XFS_ATTR_PARENT)/* * Alignment for namelist and valuelist entries (since they are mixed
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -917,6 +917,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/*
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 08/18] xfs: add parent pointer support to attribute code
Quoted 9 lines · Allison Henderson
Add the new parent attribute type.
XFS_ATTR_PARENTis used only for parent pointer entries; it uses reserved blocks likeXFS_ATTR_ROOT.[dchinner: forward ported and cleaned up]
[achender: rebased]Signed-off-by Mark Tinguely <[email protected]>Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>
Looks good now,
--D
Quoted 61 lines
--- fs/xfs/libxfs/xfs_attr.c | 4 +++- fs/xfs/libxfs/xfs_da_format.h | 5 ++++- fs/xfs/libxfs/xfs_log_format.h | 1 + 3 files changed, 8 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_attr.c +3
-1@@ -966,11 +966,13 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; -bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool rsvd;int error, local;int rmt_blks =0 ;unsigned int total; + rsvd = (args->attr_filter & (XFS_ATTR_ROOT |XFS_ATTR_PARENT )) !=0 ; +if (xfs_is_shutdown (dp->i_mount))return -EIO ;fs/xfs/libxfs/xfs_da_format.h +4
-1@@ -688,12 +688,15 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */ #define XFS_ATTR_SECURE_BIT 2/* limit access to secure attrs */ +#define XFS_ATTR_PARENT_BIT 3/* parent pointer attrs */ #define XFS_ATTR_INCOMPLETE_BIT 7/* attr in middle of create/delete */ #define XFS_ATTR_LOCAL (1u << XFS_ATTR_LOCAL_BIT)#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT) +#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_NSP_ONDISK_MASK \ + (XFS_ATTR_ROOT | XFS_ATTR_SECURE | XFS_ATTR_PARENT)/* * Alignment for namelist and valuelist entries (since they are mixed fs/xfs/libxfs/xfs_log_format.h +1
-0@@ -917,6 +917,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/* Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 08/18] xfs: add parent pointer support to attribute code
Quoted 8 lines · Darrick J. Wong
Quoted 7 lines · Allison Henderson
Add the new parent attribute type.
XFS_ATTR_PARENTis used only for parent pointer
entries; it uses reserved blocks likeXFS_ATTR_ROOT.[dchinner: forward ported and cleaned up]
[achender: rebased]
Quoted 6 lines
Quoted 3 lines
Signed-off-by Mark Tinguely <[email protected]>Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>Looks good now,
Reviewed-by Darrick J. Wong <[email protected]>
Great! Thanks!
Allison
Quoted 75 lines
--D
Quoted 72 lines
--- fs/xfs/libxfs/xfs_attr.c | 4 +++- fs/xfs/libxfs/xfs_da_format.h | 5 ++++- fs/xfs/libxfs/xfs_log_format.h | 1 + 3 files changed, 8 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_attr.c +3
-1@@ -966,11 +966,13 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; -bool rsvd = (args->attr_filter & XFS_ATTR_ROOT); +bool rsvd;int error, local;int rmt_blks =0 ;unsigned int total; + rsvd = (args->attr_filter & (XFS_ATTR_ROOT |XFS_ATTR_PARENT )) !=0 ; +if (xfs_is_shutdown (dp->i_mount))fs/xfs/libxfs/xfs_attr.c +0
-0return -EIO ;fs/xfs/libxfs/xfs_da_format.h +0
-0@@ -688,12 +688,15 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */
#defineXFS_ATTR_SECURE_BIT2 /* limit access to secure
attrs */
+#defineXFS_ATTR_PARENT_BIT3 /* parent pointer attrs */
#defineXFS_ATTR_INCOMPLETE_BIT7 /* attr in middle
of create/delete */
#defineXFS_ATTR_LOCAL(1u <<XFS_ATTR_LOCAL_BIT)
#defineXFS_ATTR_ROOT(1u <<XFS_ATTR_ROOT_BIT)
#defineXFS_ATTR_SECURE(1u <<XFS_ATTR_SECURE_BIT)fs/xfs/libxfs/xfs_da_format.h +1
-1+#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT |
XFS_ATTR_SECURE)fs/xfs/libxfs/xfs_da_format.h +2
-0+#define XFS_ATTR_NSP_ONDISK_MASK \ + (XFS_ATTR_ROOT | XFS_ATTR_SECURE |
XFS_ATTR_PARENT)/*
* Alignment for namelist and valuelist entries (since they are
mixedfs/xfs/libxfs/xfs_log_format.h +1
-0@@ -917,6 +917,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/* Signature · 1 line
2.25.1
PATCH RESEND v2 9/18 xfs: define parent pointer xattr format
In reply to Allison HendersonWe need to define the parent pointer attribute format before we start adding support for it into all the code that needs to use it. The EA format we will use encodes the following information:
name={parent inode #, parent inode generation, dirent offset}
value={dirent filename}The inode/gen gives all the information we need to reliably identify the parent without requiring child->parent lock ordering, and allows userspace to do pathname component level reconstruction without the kernel ever needing to verify the parent itself as part of ioctl calls.
By using the dirent offset in the EA name, we have a method of knowing the exact parent pointer EA we need to modify/remove in rename/unlink without an unbound EA name search.
By keeping the dirent name in the value, we have enough information to be able to validate and reconstruct damaged directory trees. While the diroffset of a filename alone is not unique enough to identify the child, the {diroffset,filename,child_inode} tuple is sufficient. That is, if the diroffset gets reused and points to a different filename, we can detect that from the contents of EA. If a link of the same name is created, then we can check whether it points at the same inode as the parent EA we current have.
[achender: rebased, changed __unint32_t to xfs_dir2_dataptr_t,
changed p_ino to xfs_ino_t and p_namelen to uint8_t, moved to xfs_da_format for xfs_dir2_dataptr_t]
--- fs/xfs/libxfs/xfs_da_format.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
fs/xfs/libxfs/xfs_da_format.h +25 -0
@@ -805,4 +805,29 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); +/* +* Parent pointer attribute format definition +* +* EA name encodes the parent inode number, generation and the offset of +* the dirent that points to the child inode. The EA value contains the +* same name as the dirent in the parent directory. +*/ +struct xfs_parent_name_rec { +__be64 p_ino; +__be32 p_gen; +__be32 p_diroffset; +}; + +/* +* incore version of the above, also contains name pointers so callers +* can pass/obtain all the parent pointer information in a single structure +*/ +struct xfs_parent_name_irec { +xfs_ino_t p_ino; +uint32_t p_gen; +xfs_dir2_dataptr_t p_diroffset; +const char *p_name; +uint8_t p_namelen; +}; +#endif /* __XFS_DA_FORMAT_H__ */
Signature · 1 line
2.25.1
PATCH RESEND v2 10/18 xfs: Add xfs_verify_pptr
In reply to Allison HendersonAttribute names of parent pointers are not strings. So we need to modify attr_namecheck to verify parent pointer records when the XFS_ATTR_PARENT flag is set.
--- fs/xfs/libxfs/xfs_attr.c | 43 +++++++++++++++++++++++++++++++++++++--- fs/xfs/libxfs/xfs_attr.h | 3 ++- fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_attr_item.c | 6 ++++-- fs/xfs/xfs_attr_list.c | 17 +++++++++++----- 5 files changed, 59 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_attr.c +40 -3
@@ -1567,9 +1567,29 @@ xfs_attr_node_get(return error; } -/* Returns true if the attribute entry name is valid. */ -bool -xfs_attr_namecheck ( +/* +* Verify parent pointer attribute is valid. +* Return true on success or false on failure +*/ +STATIC bool +xfs_verify_pptr (struct xfs_mount *mp,struct xfs_parent_name_rec *rec) +{ +xfs_ino_t p_ino = (xfs_ino_t )be64_to_cpu (rec->p_ino); +xfs_dir2_dataptr_t p_diroffset = + (xfs_dir2_dataptr_t )be32_to_cpu (rec->p_diroffset); + +if (!xfs_verify_ino (mp, p_ino)) +return false; + +if (p_diroffset >XFS_DIR2_MAX_DATAPTR ) +return false; + +return true; +} + +/* Returns true if the string attribute entry name is valid. */ +static bool +xfs_str_attr_namecheck (const void *name,size_t length) {
@@ -1584,6 +1604,23 @@ xfs_attr_namecheck(return !memchr (name,0 , length); } +/* Returns true if the attribute entry name is valid. */ +bool +xfs_attr_namecheck ( +struct xfs_mount *mp, +const void *name, +size_t length, +int flags) +{ +if (flags &XFS_ATTR_PARENT ) { +if (length !=sizeof (struct xfs_parent_name_rec )) +return false; +return xfs_verify_pptr (mp, (struct xfs_parent_name_rec *)name); + } + +return xfs_str_attr_namecheck (name, length); +} +int __init xfs_attr_intent_init_cache (void ) {
fs/xfs/libxfs/xfs_attr.h +2 -1
@@ -547,7 +547,8 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, +int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);
fs/xfs/scrub/attr.c +1 -1
@@ -128,7 +128,7 @@ xchk_xattr_listent( }/* Does this name make sense? */ -if (!xfs_attr_namecheck (name, namelen)) { +if (!xfs_attr_namecheck (sx->sc->mp, name, namelen, flags)) {xchk_fblock_set_corrupt (sx->sc,XFS_ATTR_FORK , args.blkno);return ; }
fs/xfs/xfs_attr_item.c +4 -2
@@ -587,7 +587,8 @@ xfs_attri_item_recover(*/ attrp = &attrip->attri_format;if (!xfs_attri_validate (mp, attrp) || - !xfs_attr_namecheck (nv->name.i_addr, nv->name.i_len)) + !xfs_attr_namecheck (mp, nv->name.i_addr, nv->name.i_len, + attrp->alfi_attr_filter))return -EFSCORRUPTED ; error =xlog_recover_iget (mp, attrp->alfi_ino, &ip);
@@ -727,7 +728,8 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } -if (!xfs_attr_namecheck (attr_name, attri_formatp->alfi_name_len)) { +if (!xfs_attr_namecheck (mp, attr_name, attri_formatp->alfi_name_len, + attri_formatp->alfi_attr_filter)) {XFS_ERROR_REPORT (__func__,XFS_ERRLEVEL_LOW , mp);return -EFSCORRUPTED ; }
fs/xfs/xfs_attr_list.c +12 -5
@@ -58,9 +58,13 @@ xfs_attr_shortform_list(struct xfs_attr_sf_sort *sbuf, *sbp;struct xfs_attr_shortform *sf;struct xfs_attr_sf_entry *sfe; +struct xfs_mount *mp;int sbsize, nsbuf, count, i;int error =0 ; +ASSERT (context !=NULL ); +ASSERT (dp !=NULL ); + mp = dp->i_mount; sf = (struct xfs_attr_shortform *)dp->i_af.if_u1.if_data;ASSERT (sf !=NULL );if (!sf->hdr.count)
@@ -82,8 +86,9 @@ xfs_attr_shortform_list( (dp->i_af.if_bytes + sf->hdr.count * 16) < context->bufsize)) {for (i =0 , sfe = &sf->list[0 ]; i < sf->hdr.count; i++) {if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sfe->nameval, - sfe->namelen))) + !xfs_attr_namecheck (mp, sfe->nameval, + sfe->namelen, + sfe->flags)))return -EFSCORRUPTED ; context->put_listent(context, sfe->flags,
@@ -174,8 +179,9 @@ xfs_attr_shortform_list( cursor->offset =0 ; }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sbp->name, - sbp->namelen))) { + !xfs_attr_namecheck (mp, sbp->name, + sbp->namelen, + sbp->flags))) { error = -EFSCORRUPTED ; goto out; }
@@ -465,7 +471,8 @@ xfs_attr3_leaf_list_int( }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (name, namelen))) + !xfs_attr_namecheck (mp, name, namelen, + entry->flags)))return -EFSCORRUPTED ; context->put_listent (context, entry->flags, name, namelen, valuelen);
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 10/18] xfs: Add xfs_verify_pptr
Quoted 25 lines · Allison Henderson
Attribute names of parent pointers are not strings. So we need to modify
attr_namecheckto verify parent pointer records when theXFS_ATTR_PARENTflag is set.Signed-off-by Allison Henderson <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>--- fs/xfs/libxfs/xfs_attr.c | 43 +++++++++++++++++++++++++++++++++++++--- fs/xfs/libxfs/xfs_attr.h | 3 ++- fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_attr_item.c | 6 ++++-- fs/xfs/xfs_attr_list.c | 17 +++++++++++----- 5 files changed, 59 insertions(+), 12 deletions(-)fs/xfs/libxfs/xfs_attr.c +0
-3@@ -1567,9 +1567,29 @@ xfs_attr_node_get(return error; } -/* Returns true if the attribute entry name is valid. */ -bool -xfs_attr_namecheck(
fs/xfs/libxfs/xfs_attr.c xfs_attr_node_get( · line 1579
Quoted 10 lines
fs/xfs/libxfs/xfs_attr.c +10
-0+/* +* Verify parent pointer attribute is valid. +* Return true on success or false on failure +*/ +STATIC bool +xfs_verify_pptr (struct xfs_mount *mp,struct xfs_parent_name_rec *rec) +{ +xfs_ino_t p_ino = (xfs_ino_t )be64_to_cpu (rec->p_ino); +xfs_dir2_dataptr_t p_diroffset = + (xfs_dir2_dataptr_t )be32_to_cpu (rec->p_diroffset);
I guess I should complain about the indentation here...
STATIC bool
xfs_verify_pptr(
struct xfs_mount *mp,
struct xfs_parent_name_rec *rec)
{
xfs_ino_t p_ino;
xfs_dir2_dataptr_t p_diroffset;
p_ino = be64_to_cpu(rec->p_ino);
p_diroffset = be32_to_cpu(rec->p_diroffset);(You can keep the RVB tag if you clean this up for the next revision.)
--D
Quoted 146 lines
fs/xfs/libxfs/xfs_attr.c +30
-0+ +if (!xfs_verify_ino (mp, p_ino)) +return false; + +if (p_diroffset >XFS_DIR2_MAX_DATAPTR ) +return false; + +return true; +} + +/* Returns true if the string attribute entry name is valid. */ +static bool +xfs_str_attr_namecheck (const void *name,size_t length) {@@ -1584,6 +1604,23 @@ xfs_attr_namecheck(return !memchr (name,0 , length); } +/* Returns true if the attribute entry name is valid. */ +bool +xfs_attr_namecheck ( +struct xfs_mount *mp, +const void *name, +size_t length, +int flags) +{ +if (flags &XFS_ATTR_PARENT ) { +if (length !=sizeof (struct xfs_parent_name_rec )) +return false; +return xfs_verify_pptr (mp, (struct xfs_parent_name_rec *)name); + } + +return xfs_str_attr_namecheck (name, length); +} +int __init xfs_attr_intent_init_cache (void ) {fs/xfs/libxfs/xfs_attr.h +2
-1@@ -547,7 +547,8 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, +int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);fs/xfs/scrub/attr.c +1
-1@@ -128,7 +128,7 @@ xchk_xattr_listent( }/* Does this name make sense? */ -if (!xfs_attr_namecheck (name, namelen)) { +if (!xfs_attr_namecheck (sx->sc->mp, name, namelen, flags)) {xchk_fblock_set_corrupt (sx->sc,XFS_ATTR_FORK , args.blkno);return ; }fs/xfs/xfs_attr_item.c +4
-2@@ -587,7 +587,8 @@ xfs_attri_item_recover(*/ attrp = &attrip->attri_format;if (!xfs_attri_validate (mp, attrp) || - !xfs_attr_namecheck (nv->name.i_addr, nv->name.i_len)) + !xfs_attr_namecheck (mp, nv->name.i_addr, nv->name.i_len, + attrp->alfi_attr_filter))return -EFSCORRUPTED ; error =xlog_recover_iget (mp, attrp->alfi_ino, &ip);@@ -727,7 +728,8 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } -if (!xfs_attr_namecheck (attr_name, attri_formatp->alfi_name_len)) { +if (!xfs_attr_namecheck (mp, attr_name, attri_formatp->alfi_name_len, + attri_formatp->alfi_attr_filter)) {XFS_ERROR_REPORT (__func__,XFS_ERRLEVEL_LOW , mp);return -EFSCORRUPTED ; }fs/xfs/xfs_attr_list.c +12
-5@@ -58,9 +58,13 @@ xfs_attr_shortform_list(struct xfs_attr_sf_sort *sbuf, *sbp;struct xfs_attr_shortform *sf;struct xfs_attr_sf_entry *sfe; +struct xfs_mount *mp;int sbsize, nsbuf, count, i;int error =0 ; +ASSERT (context !=NULL ); +ASSERT (dp !=NULL ); + mp = dp->i_mount; sf = (struct xfs_attr_shortform *)dp->i_af.if_u1.if_data;ASSERT (sf !=NULL );if (!sf->hdr.count)@@ -82,8 +86,9 @@ xfs_attr_shortform_list( (dp->i_af.if_bytes + sf->hdr.count * 16) < context->bufsize)) {for (i =0 , sfe = &sf->list[0 ]; i < sf->hdr.count; i++) {if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sfe->nameval, - sfe->namelen))) + !xfs_attr_namecheck (mp, sfe->nameval, + sfe->namelen, + sfe->flags)))return -EFSCORRUPTED ; context->put_listent(context, sfe->flags,@@ -174,8 +179,9 @@ xfs_attr_shortform_list( cursor->offset =0 ; }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sbp->name, - sbp->namelen))) { + !xfs_attr_namecheck (mp, sbp->name, + sbp->namelen, + sbp->flags))) { error = -EFSCORRUPTED ; goto out; }@@ -465,7 +471,8 @@ xfs_attr3_leaf_list_int( }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (name, namelen))) + !xfs_attr_namecheck (mp, name, namelen, + entry->flags)))return -EFSCORRUPTED ; context->put_listent (context, entry->flags, name, namelen, valuelen);Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 10/18] xfs: Add xfs_verify_pptr
Quoted 54 lines · Darrick J. Wong
Quoted 29 lines · Allison Henderson
Attribute names of parent pointers are not strings. So we need to modify
attr_namecheckto verify parent pointer records when theXFS_ATTR_PARENTflag is
set.Signed-off-by Allison Henderson <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>--- fs/xfs/libxfs/xfs_attr.c | 43 +++++++++++++++++++++++++++++++++++++--- fs/xfs/libxfs/xfs_attr.h | 3 ++- fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_attr_item.c | 6 ++++-- fs/xfs/xfs_attr_list.c | 17 +++++++++++----- 5 files changed, 59 insertions(+), 12 deletions(-)fs/xfs/libxfs/xfs_attr.c +1
-3@@ -1567,9 +1567,29 @@ xfs_attr_node_get(return error; } -/* Returns true if the attribute entry name is valid. */ -bool -xfs_attr_namecheck( +/* fs/xfs/libxfs/xfs_attr.c xfs_attr_node_get( · line 1580
Quoted 10 lines
fs/xfs/libxfs/xfs_attr.c +9
-0+* Verify parent pointer attribute is valid. +* Return true on success or false on failure +*/ +STATIC bool +xfs_verify_pptr (struct xfs_mount *mp,struct xfs_parent_name_rec *rec) +{ +xfs_ino_t p_ino = (xfs_ino_t )be64_to_cpu (rec->p_ino); +xfs_dir2_dataptr_t p_diroffset = + (xfs_dir2_dataptr_t )be32_to_cpu (rec->p_diroffset);I guess I should complain about the indentation here...
STATIC bool
xfs_verify_pptr(
structxfs_mount*mp,
structxfs_parent_name_rec*rec){ xfs_ino_t p_ino; xfs_dir2_dataptr_t p_diroffset; p_ino = be64_to_cpu(rec->p_ino); p_diroffset = be32_to_cpu(rec->p_diroffset);
Quoted 2 lines
(You can keep the RVB tag if you clean this up for the next revision.)
Sure, will fix
Allison
Quoted 164 lines
--D
Quoted 161 lines
fs/xfs/libxfs/xfs_attr.c +13
-0+ +if (!xfs_verify_ino (mp, p_ino)) +return false; + +if (p_diroffset >XFS_DIR2_MAX_DATAPTR ) +return false; + +return true; +} + +/* Returns true if the string attribute entry name is valid. */ +static bool +xfs_str_attr_namecheck (const void *name,size_t length){
fs/xfs/libxfs/xfs_attr.c +17
-0@@ -1584,6 +1604,23 @@ xfs_attr_namecheck(return !memchr (name,0 , length); } +/* Returns true if the attribute entry name is valid. */ +bool +xfs_attr_namecheck ( +struct xfs_mount *mp, +const void *name, +size_t length, +int flags) +{ +if (flags &XFS_ATTR_PARENT ) { +if (length !=sizeof (struct xfs_parent_name_rec )) +return false; +return xfs_verify_pptr (mp, (struct xfs_parent_name_rec *)name); + } + +return xfs_str_attr_namecheck (name, length); +} +int __init xfs_attr_intent_init_cache (void ){
fs/xfs/libxfs/xfs_attr.h +2
-1@@ -547,7 +547,8 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, +int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local); void xfs_init_attr_trans(struct xfs_da_args *args, struct
xfs_trans_res*tres,fs/xfs/libxfs/xfs_attr.h +0
-0unsigned int *total);fs/xfs/scrub/attr.c +1
-1@@ -128,7 +128,7 @@ xchk_xattr_listent( }/* Does this name make sense? */ -if (!xfs_attr_namecheck (name, namelen)) { +if (!xfs_attr_namecheck (sx->sc->mp, name, namelen, flags)) {xchk_fblock_set_corrupt (sx->sc,XFS_ATTR_FORK , args.blkno);return ;fs/xfs/scrub/attr.c +0
-0}fs/xfs/xfs_attr_item.c +4
-2@@ -587,7 +587,8 @@ xfs_attri_item_recover(*/ attrp = &attrip->attri_format;if (!xfs_attri_validate (mp, attrp) || - !xfs_attr_namecheck (nv->name.i_addr, nv->name.i_len)) + !xfs_attr_namecheck (mp, nv->name.i_addr, nv->name.i_len, + attrp->alfi_attr_filter))return -EFSCORRUPTED ; error =xlog_recover_iget (mp, attrp->alfi_ino, &ip);@@ -727,7 +728,8 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } -if (!xfs_attr_namecheck (attr_name, attri_formatp- >alfi_name_len)) { +if (!xfs_attr_namecheck (mp, attr_name, attri_formatp- >alfi_name_len, + attri_formatp->alfi_attr_filter)) {XFS_ERROR_REPORT (__func__,XFS_ERRLEVEL_LOW , mp);fs/xfs/xfs_attr_item.c +0
-0return -EFSCORRUPTED ; }fs/xfs/xfs_attr_list.c +4
-2@@ -58,9 +58,13 @@ xfs_attr_shortform_list(struct xfs_attr_sf_sort *sbuf, *sbp;struct xfs_attr_shortform *sf;struct xfs_attr_sf_entry *sfe; +struct xfs_mount *mp;int sbsize, nsbuf, count, i;int error =0 ; +ASSERT (context !=NULL ); +ASSERT (dp !=NULL ); + mp = dp->i_mount; sf = (struct xfs_attr_shortform *)dp->i_af.if_u1.if_data;ASSERT (sf !=NULL );if (!sf->hdr.count)@@ -82,8 +86,9 @@ xfs_attr_shortform_list( (dp->i_af.if_bytes + sf->hdr.count * 16) < context- >bufsize)) {for (i =0 , sfe = &sf->list[0 ]; i < sf->hdr.count; i++) { if (XFS_IS_CORRUPT(context->dp->i_mount, - !xfs_attr_namecheck(sfe- >nameval, - sfe-Quoted 1 line
namelen)))
fs/xfs/xfs_attr_list.c +1
-0+ !xfs_attr_namecheck(mp, sfe-Quoted 1 line
nameval,
fs/xfs/xfs_attr_list.c +1
-0+ sfe-Quoted 1 line
namelen,
fs/xfs/xfs_attr_list.c +1
-0+ sfe-Quoted 1 line
flags)))
fs/xfs/xfs_attr_list.c +3
-2return -EFSCORRUPTED ; context->put_listent(context, sfe->flags,@@ -174,8 +179,9 @@ xfs_attr_shortform_list( cursor->offset =0 ; } if (XFS_IS_CORRUPT(context->dp->i_mount, - !xfs_attr_namecheck (sbp->name, - sbp->namelen))) { + !xfs_attr_namecheck(mp, sbp->name, + sbp->namelen, + sbp->flags))) { error = -EFSCORRUPTED; goto out;fs/xfs/xfs_attr_list.c +2
-1}@@ -465,7 +471,8 @@ xfs_attr3_leaf_list_int( }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (name, namelen))) + !xfs_attr_namecheck (mp, name, namelen, + entry->flags)))return -EFSCORRUPTED ; context->put_listent(context, entry->flags,name, namelen, valuelen);
Signature · 1 line
2.25.1
PATCH RESEND v2 11/18 xfs: extend transaction reservations for parent attributes
In reply to Allison HendersonWe need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.
[achender: rebased]
--- fs/xfs/libxfs/xfs_trans_resv.c | 105 +++++++++++++++++++++++++++------ 1 file changed, 86 insertions(+), 19 deletions(-)
fs/xfs/libxfs/xfs_trans_resv.c +86 -19
@@ -909,24 +909,67 @@ xfs_calc_sb_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize); } -void -xfs_trans_resv_calc ( -struct xfs_mount *mp, -struct xfs_trans_resv *resp) +STATIC void +xfs_calc_parent_ptr_reservations ( +struct xfs_mount *mp) { -int logcount_adj =0 ; +struct xfs_trans_resv *resp =M_RES (mp); -/* -* The following transactions are logged in physical format and -* require a permanent reservation on space. -*/ - resp->tr_write.tr_logres =xfs_calc_write_reservation (mp, false); - resp->tr_write.tr_logcount =XFS_WRITE_LOG_COUNT ; - resp->tr_write.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +/* Calculate extra space needed for parent pointer attributes */ +if (!xfs_has_parent (mp)) +return ; - resp->tr_itruncate.tr_logres =xfs_calc_itruncate_reservation (mp, false); - resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; - resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +/* rename can add/remove/modify 4 parent attributes */ + resp->tr_rename.tr_logres +=4 *max (resp->tr_attrsetm.tr_logres, + resp->tr_attrrm.tr_logres); + resp->tr_rename.tr_logcount +=4 *max (resp->tr_attrsetm.tr_logcount, + resp->tr_attrrm.tr_logcount); + +/* create will add 1 parent attribute */ + resp->tr_create.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_create.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* mkdir will add 1 parent attribute */ + resp->tr_mkdir.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_mkdir.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* link will add 1 parent attribute */ + resp->tr_link.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_link.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* symlink will add 1 parent attribute */ + resp->tr_symlink.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_symlink.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* remove will remove 1 parent attribute */ + resp->tr_remove.tr_logres += resp->tr_attrrm.tr_logres; + resp->tr_remove.tr_logcount += resp->tr_attrrm.tr_logcount; +} + +/* +* Namespace reservations. +* +* These get tricky when parent pointers are enabled as we have attribute +* modifications occurring from within these transactions. Rather than confuse +* each of these reservation calculations with the conditional attribute +* reservations, add them here in a clear and concise manner. This assumes that +* the attribute reservations have already been calculated. +* +* Note that we only include the static attribute reservation here; the runtime +* reservation will have to be modified by the size of the attributes being +* added/removed/modified. See the comments on the attribute reservation +* calculations for more details. +* +* Note for rename: rename will vastly overestimate requirements. This will be +* addressed later when modifications are made to ensure parent attribute +* modifications can be done atomically with the rename operation. +*/ +STATIC void +xfs_calc_namespace_reservations ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +ASSERT (resp->tr_attrsetm.tr_logres >0 ); resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); resp->tr_rename.tr_logcount =XFS_RENAME_LOG_COUNT ;
@@ -948,15 +991,37 @@ xfs_trans_resv_calc( resp->tr_create.tr_logcount =XFS_CREATE_LOG_COUNT ; resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); + resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; + resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + +xfs_calc_parent_ptr_reservations (mp); +} + +void +xfs_trans_resv_calc ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +int logcount_adj =0 ; + +/* +* The following transactions are logged in physical format and +* require a permanent reservation on space. +*/ + resp->tr_write.tr_logres =xfs_calc_write_reservation (mp, false); + resp->tr_write.tr_logcount =XFS_WRITE_LOG_COUNT ; + resp->tr_write.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_itruncate.tr_logres =xfs_calc_itruncate_reservation (mp, false); + resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; + resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + resp->tr_create_tmpfile.tr_logres =xfs_calc_create_tmpfile_reservation (mp); resp->tr_create_tmpfile.tr_logcount =XFS_CREATE_TMPFILE_LOG_COUNT ; resp->tr_create_tmpfile.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); - resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; - resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_ifree.tr_logres =xfs_calc_ifree_reservation (mp); resp->tr_ifree.tr_logcount =XFS_INACTIVE_LOG_COUNT ; resp->tr_ifree.tr_logflags |=XFS_TRANS_PERM_LOG_RES ;
@@ -986,6 +1051,8 @@ xfs_trans_resv_calc( resp->tr_qm_dqalloc.tr_logcount =XFS_WRITE_LOG_COUNT ; resp->tr_qm_dqalloc.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +xfs_calc_namespace_reservations (mp, resp); +/* * The following transactions are logged in logical format with * a default log count.
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 11/18] xfs: extend transaction reservations for parent attributes
Quoted 6 lines · Allison Henderson
We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.
While we're on the topic of log reservations ... Dave and I noticed during the 5.19 cycle that xfs_log_calc_max_attrsetm_res has a unit conversion problem when it's trying to compute the minimum log size:
STATIC int
xfs_log_calc_max_attrsetm_res(
struct xfs_mount *mp)
{
int size;
int nblks;
size = xfs_attr_leaf_entsize_local_max(mp->m_attr_geo->blksize) -
MAXNAMELEN - 1;Notice here that @size is the maximum amount of space that a local format attribute can use in an xattr leaf block. The computation is in units of bytes.
nblks = XFS_DAENTER_SPACE_RES(mp, XFS_ATTR_FORK); nblks += XFS_B_TO_FSB(mp, size);
...and here we convert bytes to fs blocks for the block count computation...
nblks += XFS_NEXTENTADD_SPACE_RES(mp, size, XFS_ATTR_FORK);
...but here we pass the byte count into a macro that takes a block count as its second parameter and returns the number of bmbt blocks needed to add that many blocks to an attribute fork. Oops!
I would like to fix this incorrect code, but it's never a good idea to adjust downwards the min log size calculation for existing filesystems, because this can result in the situation where new mkfs formats a filesystem with a small enough log that an old kernel won't mount it.
Therefore, the corrected logic would have to be gated on whatever happens to be the next new ondisk feature. It's probably too late to do this for large extent counts, but fixing the calculation would be (I think) appropriate for parent pointers, since it's still undergoing review and won't be an easy upgrade, which eliminates the legacy problem.
I'll attach the patches that I've written as patches 19 and 20 to this patchset, if you don't mind having a look and adding them?
return M_RES(mp)->tr_attrsetm.tr_logres + M_RES(mp)->tr_attrsetrt.tr_logres * nblks; }
Quoted 25 lines
[achender: rebased]
Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/libxfs/xfs_trans_resv.c | 105 +++++++++++++++++++++++++++------ 1 file changed, 86 insertions(+), 19 deletions(-)fs/xfs/libxfs/xfs_trans_resv.c +3
-5@@ -909,24 +909,67 @@ xfs_calc_sb_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize); } -void -xfs_trans_resv_calc ( -struct xfs_mount *mp, -struct xfs_trans_resv *resp) +STATIC void +xfs_calc_parent_ptr_reservations ( +struct xfs_mount *mp) { -int logcount_adj =0 ;
fs/xfs/libxfs/xfs_trans_resv.c xfs_calc_sb_reservation( · line 918
Quoted 10 lines
fs/xfs/libxfs/xfs_trans_resv.c +2
-7+struct xfs_trans_resv *resp =M_RES (mp); -/* -* The following transactions are logged in physical format and -* require a permanent reservation on space. -*/ - resp->tr_write.tr_logres =xfs_calc_write_reservation (mp, false); - resp->tr_write.tr_logcount =XFS_WRITE_LOG_COUNT ; - resp->tr_write.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +/* Calculate extra space needed for parent pointer attributes */
This might be better expressed as a comment just prior to the function declaration above.
fs/xfs/libxfs/xfs_trans_resv.c xfs_calc_sb_reservation( · line 924
Quoted 9 lines
fs/xfs/libxfs/xfs_trans_resv.c +5
-3+if (!xfs_has_parent (mp)) +return ; - resp->tr_itruncate.tr_logres =xfs_calc_itruncate_reservation (mp, false); - resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; - resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +/* rename can add/remove/modify 4 parent attributes */ + resp->tr_rename.tr_logres +=4 *max (resp->tr_attrsetm.tr_logres, + resp->tr_attrrm.tr_logres);
Why does the per-transaction reservation increase by 4x the amount of space needed to set (or delete) an xattr? The pptr patchset now uses logged xattrs, which means that each xattr update needed to commit the rename operation will happen in a separate transaction. IOWs, each transaction in the chain does not have to handle *every* update that must be made during the entire chain, it only has to handle one step of the full process.
Doesn't that mean that the size of tr_rename.tr_logres only needs to increase by the amount of space needed to log the four(?) xattr items to the first transaction in the chain? AFAICT, it also can't be smaller than max(resp->tr_attrsetm.tr_logres, resp->tr_attrrm.tr_logres);
(I'm also not sure why four -- the patch for xfs_rename only creates three xfs_parent_defer objects.)
I also think that adjusting tr_rename to account for parent pointers is something that should be done in xfs_calc_rename_reservation, not a separate function:
/*
* In renaming a files we can modify (t1):
* the four inodes involved: 4 * inode size * the two directory btrees: 2 * (max depth + v2) * dir block size * the two directory bmap btrees: 2 * max depth * block size * And the bmap_finish transaction can free dir and bmap blocks (two sets * of bmap blocks) giving (t2): * the agf for the ags in which the blocks live: 3 * sector size * the agfl for the ags in which the blocks live: 3 * sector size * the superblock for the free block count: sector size * the allocation btrees: 3 exts * 2 trees * (2 * max depth - 1) * block size * If parent pointers are enabled (t3), then each transaction in the chain * must be capable of setting or removing the extended attribute * containing the parent information. It must also be able to handle * the three xattr intent items that track the progress of the parent * pointer update.
*/
STATIC uint
xfs_calc_rename_reservation(
struct xfs_mount *mp)
{
unsigned int overhead = XFS_DQUOT_LOGRES(mp);
unsigned int t1, t2, t3 = 0;
t1 = xfs_calc_inode_res(mp, 4) +
xfs_calc_buf_res(2 * XFS_DIROP_LOG_COUNT(mp),
XFS_FSB_TO_B(mp, 1));
t2 = xfs_calc_buf_res(7, mp->m_sb.sb_sectsize) +
xfs_calc_buf_res(xfs_allocfree_block_count(mp, 3),
XFS_FSB_TO_B(mp, 1))));
if (xfs_has_parent(mp)) {
t3 = max(resp->tr_attrsetm.tr_logres,
resp->tr_attrrm.tr_logres);
overhead += 3 * (size of a pptr xattr intent item);
}
return overhead + max3(t1, t2, t3);
}fs/xfs/libxfs/xfs_trans_resv.c xfs_calc_sb_reservation( · line 926
Quoted 2 lines
fs/xfs/libxfs/xfs_trans_resv.c +2
-0+ resp->tr_rename.tr_logcount +=4 *max (resp->tr_attrsetm.tr_logcount, + resp->tr_attrrm.tr_logcount);
Looks correct, module the 4 vs. 3 thing.
Quoted 28 lines
fs/xfs/libxfs/xfs_trans_resv.c +28
-0+ +/* create will add 1 parent attribute */ + resp->tr_create.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_create.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* mkdir will add 1 parent attribute */ + resp->tr_mkdir.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_mkdir.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* link will add 1 parent attribute */ + resp->tr_link.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_link.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* symlink will add 1 parent attribute */ + resp->tr_symlink.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_symlink.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* remove will remove 1 parent attribute */ + resp->tr_remove.tr_logres += resp->tr_attrrm.tr_logres; + resp->tr_remove.tr_logcount += resp->tr_attrrm.tr_logcount; +} + +/* +* Namespace reservations. +* +* These get tricky when parent pointers are enabled as we have attribute +* modifications occurring from within these transactions. Rather than confuse +* each of these reservation calculations with the conditional attribute
fs/xfs/libxfs/xfs_trans_resv.c xfs_calc_sb_reservation( · line 964
Quoted 10 lines
fs/xfs/libxfs/xfs_trans_resv.c +10
-0+* reservations, add them here in a clear and concise manner. This assumes that +* the attribute reservations have already been calculated. +* +* Note that we only include the static attribute reservation here; the runtime +* reservation will have to be modified by the size of the attributes being +* added/removed/modified. See the comments on the attribute reservation +* calculations for more details. +* +* Note for rename: rename will vastly overestimate requirements. This will be +* addressed later when modifications are made to ensure parent attribute
Later? I took a look at the rename patch, and it looks like we're using logged xattrs from the start.
--D
Quoted 65 lines
fs/xfs/libxfs/xfs_trans_resv.c +36
-4+* modifications can be done atomically with the rename operation. +*/ +STATIC void +xfs_calc_namespace_reservations ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +ASSERT (resp->tr_attrsetm.tr_logres >0 ); resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); resp->tr_rename.tr_logcount =XFS_RENAME_LOG_COUNT ;@@ -948,15 +991,37 @@ xfs_trans_resv_calc( resp->tr_create.tr_logcount =XFS_CREATE_LOG_COUNT ; resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); + resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; + resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + +xfs_calc_parent_ptr_reservations (mp); +} + +void +xfs_trans_resv_calc ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +int logcount_adj =0 ; + +/* +* The following transactions are logged in physical format and +* require a permanent reservation on space. +*/ + resp->tr_write.tr_logres =xfs_calc_write_reservation (mp, false); + resp->tr_write.tr_logcount =XFS_WRITE_LOG_COUNT ; + resp->tr_write.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_itruncate.tr_logres =xfs_calc_itruncate_reservation (mp, false); + resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; + resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + resp->tr_create_tmpfile.tr_logres =xfs_calc_create_tmpfile_reservation (mp); resp->tr_create_tmpfile.tr_logcount =XFS_CREATE_TMPFILE_LOG_COUNT ; resp->tr_create_tmpfile.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); - resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; - resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_ifree.tr_logres =xfs_calc_ifree_reservation (mp); resp->tr_ifree.tr_logcount =XFS_INACTIVE_LOG_COUNT ; resp->tr_ifree.tr_logflags |=XFS_TRANS_PERM_LOG_RES ;@@ -986,6 +1051,8 @@ xfs_trans_resv_calc( resp->tr_qm_dqalloc.tr_logcount =XFS_WRITE_LOG_COUNT ; resp->tr_qm_dqalloc.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +xfs_calc_namespace_reservations (mp, resp); +/* * The following transactions are logged in logical format with * a default log count. Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 11/18] xfs: extend transaction reservations for parent attributes
Quoted 61 lines · Darrick J. Wong
Quoted 10 lines · Allison Henderson
We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the
parent directories being modified. This means they need to be modified
in the same transaction as the parent directories, and so we need to add
the required space for the attribute modifications to the transaction
reservations.While we're on the topic of log reservations ... Dave and I noticed during the 5.19 cycle that
xfs_log_calc_max_attrsetm_reshas a unit conversion problem when it's trying to compute the minimum log size:STATIC int
xfs_log_calc_max_attrsetm_res(
structxfs_mount*mp){ int size; int nblks; size = xfs_attr_leaf_entsize_local_max(mp->m_attr_geo->blksize) - MAXNAMELEN - 1;Notice here that @size is the maximum amount of space that a local format attribute can use in an xattr leaf block. The computation is in
units of bytes.nblks = XFS_DAENTER_SPACE_RES(mp, XFS_ATTR_FORK); nblks += XFS_B_TO_FSB(mp, size);...and here we convert bytes to fs blocks for the block count computation...
nblks += XFS_NEXTENTADD_SPACE_RES(mp, size, XFS_ATTR_FORK);...but here we pass the byte count into a macro that takes a block count
as its second parameter and returns the number of bmbt blocks needed to
add that many blocks to an attribute fork. Oops!I would like to fix this incorrect code, but it's never a good idea to
adjust downwards the min log size calculation for existing filesystems,
because this can result in the situation where new mkfs formats a filesystem with a small enough log that an old kernel won't mount it.Therefore, the corrected logic would have to be gated on whatever happens to be the next new ondisk feature. It's probably too late to do
this for large extent counts, but fixing the calculation would be (I think) appropriate for parent pointers, since it's still undergoing review and won't be an easy upgrade, which eliminates the legacy problem.
Quoted 3 lines
I'll attach the patches that I've written as patches 19 and 20 to this
patchset, if you don't
Sure, I will keep an eye out for them then.
Quoted 35 lines
return M_RES(mp)->tr_attrsetm.tr_logres + M_RES(mp)->tr_attrsetrt.tr_logres * nblks; }Quoted 29 lines
[achender: rebased]
Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/libxfs/xfs_trans_resv.c | 105 +++++++++++++++++++++++++++ ------ 1 file changed, 86 insertions(+), 19 deletions(-)fs/xfs/libxfs/xfs_trans_resv.c +4
-5@@ -909,24 +909,67 @@ xfs_calc_sb_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize); } -void -xfs_trans_resv_calc ( -struct xfs_mount *mp, -struct xfs_trans_resv *resp) +STATIC void +xfs_calc_parent_ptr_reservations ( +struct xfs_mount *mp) { -int logcount_adj =0 ; +struct xfs_trans_resv *resp =M_RES (mp);
Quoted 14 lines
fs/xfs/libxfs/xfs_trans_resv.c xfs_calc_sb_reservation(
Quoted 10 lines
fs/xfs/libxfs/xfs_trans_resv.c +1
-7-/* -* The following transactions are logged in physical format and -* require a permanent reservation on space. -*/ - resp->tr_write.tr_logres =xfs_calc_write_reservation (mp, false); - resp->tr_write.tr_logcount =XFS_WRITE_LOG_COUNT ; - resp->tr_write.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +/* Calculate extra space needed for parent pointer attributes */ This might be better expressed as a comment just prior to the function
declaration above.
Alrighty, will move upwards
Quoted 11 lines
Quoted 11 lines
fs/xfs/libxfs/xfs_trans_resv.c +4
-3+if (!xfs_has_parent (mp)) +return ; - resp->tr_itruncate.tr_logres =xfs_calc_itruncate_reservation (mp, false); - resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; - resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +/* rename can add/remove/modify 4 parent attributes */ + resp->tr_rename.tr_logres += 4 * max(resp-Quoted 1 line
tr_attrsetm.tr_logres,
Quoted 11 lines
fs/xfs/libxfs/xfs_trans_resv.c xfs_calc_sb_reservation( · line 924
Quoted 1 line
fs/xfs/libxfs/xfs_trans_resv.c +1
-0+ resp->tr_attrrm.tr_logres);Why does the per-transaction reservation increase by 4x the amount of space needed to set (or delete) an xattr? The pptr patchset now uses logged xattrs, which means that each xattr update needed to commit the
rename operation will happen in a separate transaction. IOWs, each transaction in the chain does not have to handle *every* update that must be made during the entire chain, it only has to handle one step of
the full process.
Oh, I think initially this might have been pre-larp code, so probably we can just drop it now
Quoted 6 lines
Doesn't that mean that the size of tr_rename.tr_logres only needs to increase by the amount of space needed to log the four(?) xattr items to
the first transaction in the chain? AFAICT, it also can't be smaller thanmax(resp->tr_attrsetm.tr_logres, resp->tr_attrrm.tr_logres);
I think so, probably we can just leave it as max
Quoted 3 lines
(I'm also not sure why four -- the patch for
xfs_renameonly creates threexfs_parent_deferobjects.)
Hrmm, well initially I think it was for 4 inodes, but then we remembered wip in the last review, so now it's 5 right? That's why we expanded XFS_DEFER_OPS_NR_INODES in patch 1. So probably the 4 inodes needs to turn into a 5 in this patch too.
Quoted 9 lines
I also think that adjusting
tr_renameto account for parent pointers is
something that should be done inxfs_calc_rename_reservation, not a separate function:/*
* In renaming a files we can modify (t1):
* the four inodes involved: 4 * inode size
5?
Quoted 22 lines
* the two directory btrees: 2 * (max depth + v2) * dir block size
* the two directory bmap btrees: 2 * max depth * block size
* And thebmap_finishtransaction can free dir and bmap blocks (two
sets
* of bmap blocks) giving (t2):* the agf for the ags in which the blocks live: 3 * sector size * the agfl for the ags in which the blocks live: 3 * sector size * the superblock for the free block count: sector size * the allocation btrees: 3 exts * 2 trees * (2 * max depth - 1) *block size
* If parent pointers are enabled (t3), then each transaction in the
chain
* must be capable of setting or removing the extended attribute
* containing the parent information. It must also be able to
handle
* the three xattr intent items that track the progress of the
parent
* pointer update.
*/
STATIC uint
xfs_calc_rename_reservation(
structxfs_mount*mp)
Quoted 5 lines
{
unsigned int overhead = XFS_DQUOT_LOGRES(mp);
unsigned int t1, t2, t3 = 0;
t1 = xfs_calc_inode_res(mp, 4) +and then the same 5 goes in here too...
Quoted 16 lines
xfs_calc_buf_res(2 * XFS_DIROP_LOG_COUNT(mp),
XFS_FSB_TO_B(mp, 1));
t2 = xfs_calc_buf_res(7, mp->m_sb.sb_sectsize) +
xfs_calc_buf_res(xfs_allocfree_block_count(mp, 3),
XFS_FSB_TO_B(mp, 1))));
if (xfs_has_parent(mp)) {
t3 = max(resp->tr_attrsetm.tr_logres,
resp->tr_attrrm.tr_logres);
overhead += 3 * (size of a pptr xattr intent item);
}
return overhead + max3(t1, t2, t3);
}Quoted 7 lines
Quoted 4 lines
fs/xfs/libxfs/xfs_trans_resv.c +1
-0+ resp->tr_rename.tr_logcount += 4 * max(resp-Quoted 1 line
tr_attrsetm.tr_logcount,
fs/xfs/libxfs/xfs_trans_resv.c +1
-0+ resp-Quoted 1 line
tr_attrrm.tr_logcount);
Looks correct, module the 4 vs. 3 thing.
I think that looks right? 5 inodes 3 pptrs?
Quoted 43 lines
Quoted 43 lines
fs/xfs/libxfs/xfs_trans_resv.c +26
-0+ +/* create will add 1 parent attribute */ + resp->tr_create.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_create.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* mkdir will add 1 parent attribute */ + resp->tr_mkdir.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_mkdir.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* link will add 1 parent attribute */ + resp->tr_link.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_link.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* symlink will add 1 parent attribute */ + resp->tr_symlink.tr_logres += resp->tr_attrsetm.tr_logres; + resp->tr_symlink.tr_logcount += resp->tr_attrsetm.tr_logcount; + +/* remove will remove 1 parent attribute */ + resp->tr_remove.tr_logres += resp->tr_attrrm.tr_logres; + resp->tr_remove.tr_logcount += resp->tr_attrrm.tr_logcount; +} + +/* +* Namespace reservations. +* +* These get tricky when parent pointers are enabled as we have attribute
+ * modifications occurring from within these transactions. Rather than confuse
+ * each of these reservation calculations with the conditional attribute
+ * reservations, add them here in a clear and concise manner. This assumes that
+ * the attribute reservations have already been calculated.
+ *
+ * Note that we only include the static attribute reservation here; the runtime
+ * reservation will have to be modified by the size of the attributes being
+ * added/removed/modified. See the comments on the attribute reservation
+ * calculations for more details.
+ *
Quoted 8 lines
Quoted 4 lines
+ * Note for rename: rename will vastly overestimate requirements. This will be
+ * addressed later when modifications are made to ensure parent attributeLater? I took a look at the rename patch, and it looks like we're using
logged xattrs from the start.
I think it's a stale comment. Will remove. Thanks for the reviews!
Allison
Quoted 71 lines
--D
Quoted 68 lines
+ * modifications can be done atomically with the rename operation.
+ */fs/xfs/libxfs/xfs_trans_resv.c +32
-4+STATIC void +xfs_calc_namespace_reservations ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +ASSERT (resp->tr_attrsetm.tr_logres >0 ); resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); resp->tr_rename.tr_logcount =XFS_RENAME_LOG_COUNT ;@@ -948,15 +991,37 @@ xfs_trans_resv_calc( resp->tr_create.tr_logcount =XFS_CREATE_LOG_COUNT ; resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); + resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; + resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + +xfs_calc_parent_ptr_reservations (mp); +} + +void +xfs_trans_resv_calc ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +int logcount_adj =0 ; + +/* +* The following transactions are logged in physical format and +* require a permanent reservation on space. +*/ + resp->tr_write.tr_logres =xfs_calc_write_reservation (mp, false); + resp->tr_write.tr_logcount =XFS_WRITE_LOG_COUNT ; + resp->tr_write.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_itruncate.tr_logres =xfs_calc_itruncate_reservation (mp, false); + resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; + resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + resp->tr_create_tmpfile.tr_logres =xfs_calc_create_tmpfile_reservation (mp); resp->tr_create_tmpfile.tr_logcount =XFS_CREATE_TMPFILE_LOG_COUNT ; resp->tr_create_tmpfile.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); - resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; - resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; -fs/xfs/libxfs/xfs_trans_resv.c +2
-0resp->tr_ifree.tr_logres =xfs_calc_ifree_reservation (mp); resp->tr_ifree.tr_logcount =XFS_INACTIVE_LOG_COUNT ; resp->tr_ifree.tr_logflags |=XFS_TRANS_PERM_LOG_RES ;@@ -986,6 +1051,8 @@ xfs_trans_resv_calc( resp->tr_qm_dqalloc.tr_logcount =XFS_WRITE_LOG_COUNT ; resp->tr_qm_dqalloc.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +xfs_calc_namespace_reservations (mp, resp); +/* * The following transactions are logged in logical format with * a default log count. Signature · 1 line
2.25.1
PATCH RESEND v2 12/18 xfs: parent pointer attribute creation
In reply to Allison HendersonAdd parent pointer attribute during xfs_create, and subroutines to initialize attributes
[bfoster: rebase, use VFS inode generation]
[achender: rebased, changed __unint32_t to xfs_dir2_dataptr_t,
fixed some null pointer bugs,
merged error handling patch,
remove unnecessary ENOSPC handling in xfs_attr_set_first_parent]
--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_attr.h | 4 +- fs/xfs/libxfs/xfs_parent.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 34 ++++++++++ fs/xfs/xfs_inode.c | 37 ++++++++-- fs/xfs/xfs_xattr.c | 2 +- fs/xfs/xfs_xattr.h | 1 + 8 files changed, 208 insertions(+), 9 deletions(-)
fs/xfs/Makefile +1 -0
@@ -40,6 +40,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_fork.o \ xfs_inode_buf.o \ xfs_log_rlimit.o \ + xfs_parent.o \ xfs_ag_resv.o \ xfs_rmap.o \ xfs_rmap_btree.o \
fs/xfs/libxfs/xfs_attr.c +2 -2
@@ -880,7 +880,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set or remove) */
@@ -898,7 +898,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +3 -1
@@ -544,6 +544,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
@@ -552,7 +553,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/* * Check to see if the attr should be upgraded from non-existent or shortform to * single-leaf-block attribute list.
fs/xfs/libxfs/xfs_parent.c new file +134 -0
134 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h new file +34 -0
@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* +* Dynamically allocd structure used to wrap the needed data to pass around +* the defer ops machinery +*/ +struct xfs_parent_defer { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +/* +* Parent pointer attribute prototypes +*/ +void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset); +void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec); +int xfs_parent_init (xfs_mount_t *mp,xfs_inode_t *ip, +struct xfs_name *target_name, +struct xfs_parent_defer **parentp); +int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset); +void xfs_parent_cancel (xfs_mount_t *mp,struct xfs_parent_defer *parent); + +#endif /* __XFS_PARENT_H__ */
fs/xfs/xfs_inode.c +32 -5
@@ -37,6 +37,8 @@#include "xfs_reflink.h" #include "xfs_ag.h" #include "xfs_log_priv.h" +#include "xfs_parent.h" +#include "xfs_xattr.h" struct kmem_cache *xfs_inode_cache;
@@ -950,7 +952,7 @@ xfs_bumplink( int xfs_create( struct user_namespace *mnt_userns, - xfs_inode_t *dp, + struct xfs_inode *dp, struct xfs_name *name, umode_t mode, dev_t rdev,
@@ -962,7 +964,7 @@ xfs_create(struct xfs_inode *ip =NULL ;struct xfs_trans *tp =NULL ;int error; -bool unlock_dp_on_error = false; +bool unlock_dp_on_error = false;prid_t prid;struct xfs_dquot *udqp =NULL ;struct xfs_dquot *gdqp =NULL ;
@@ -970,6 +972,8 @@ xfs_create(struct xfs_trans_res *tres;uint resblks;xfs_ino_t ino; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_create (dp, name);
@@ -996,6 +1000,12 @@ xfs_create( tres = &M_RES (mp)->tr_create; } +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, dp, name, &parent); +if (error) + goto out_release_dquots; + } +/* * Initially assume that the file does not exist and * reserve the resources for that case. If that is not
@@ -1011,7 +1021,7 @@ xfs_create( resblks, &tp); }if (error) - goto out_release_dquots; + goto drop_incompat;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;
@@ -1021,6 +1031,7 @@ xfs_create(* entry pointing to them, but a directory also the "." entry * pointing to itself. */ + init_xattrs |=xfs_has_parent (mp); error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error = xfs_init_new_inode(mnt_userns, tp, dp, ino, mode,
@@ -1035,11 +1046,12 @@ xfs_create(* the transaction cancel unlocking dp so don't do it explicitly in the * error path. */ -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); unlock_dp_on_error = false; error =xfs_dir_createname (tp, dp, name, ip->i_ino, - resblks -XFS_IALLOC_SPACE_RES (mp),NULL ); + resblks -XFS_IALLOC_SPACE_RES (mp), + &diroffset);if (error) {ASSERT (error != -ENOSPC ); goto out_trans_cancel;
@@ -1055,6 +1067,17 @@ xfs_create(xfs_bumplink (tp, dp); } +/* +* If we have parent pointers, we need to add the attribute containing +* the parent information now. +*/ +if (parent) { + parent->args.dp = ip; + error =xfs_parent_defer_add (tp, dp, parent, diroffset); +if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * create transaction goes to disk before returning to
@@ -1080,6 +1103,7 @@ xfs_create( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT );return 0 ; out_trans_cancel:
@@ -1094,6 +1118,9 @@ xfs_create(xfs_finish_inode_setup (ip);xfs_irele (ip); } + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);
fs/xfs/xfs_xattr.c +1 -1
@@ -27,7 +27,7 @@* they must release the permission by calling xlog_drop_incompat_feat * when they're done. */ -static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {
fs/xfs/xfs_xattr.h +1 -0
@@ -7,6 +7,7 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *xfs_xattr_handlers[];
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 12/18] xfs: parent pointer attribute creation
Quoted 5 lines · Allison Henderson
Add parent pointer attribute during
xfs_create, and subroutines to initialize attributes[bfoster: rebase, use VFS inode generation] [achender: rebased, changed
__unint32_ttoxfs_dir2_dataptr_t,
Nit: uint32_t, not unint32_t.
Quoted 111 lines
fixed some null pointer bugs,
merged error handling patch,
remove unnecessary ENOSPC handling inxfs_attr_set_first_parent]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_attr.h | 4 +- fs/xfs/libxfs/xfs_parent.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 34 ++++++++++ fs/xfs/xfs_inode.c | 37 ++++++++-- fs/xfs/xfs_xattr.c | 2 +- fs/xfs/xfs_xattr.h | 1 + 8 files changed, 208 insertions(+), 9 deletions(-)fs/xfs/Makefile +1
-0@@ -40,6 +40,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_fork.o \ xfs_inode_buf.o \ xfs_log_rlimit.o \ + xfs_parent.o \ xfs_ag_resv.o \ xfs_rmap.o \ xfs_rmap_btree.o \fs/xfs/libxfs/xfs_attr.c +2
-2@@ -880,7 +880,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set or remove) */@@ -898,7 +898,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {fs/xfs/libxfs/xfs_attr.h +3
-1@@ -544,6 +544,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);@@ -552,7 +553,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/* * Check to see if the attr should be upgraded from non-existent or shortform to * single-leaf-block attribute list. fs/xfs/libxfs/xfs_parent.c new file +32
-0@@ -0,0 +1,134 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All rights reserved. +*/ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_da_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_bmap_btree.h" +#include "xfs_inode.h" +#include "xfs_error.h" +#include "xfs_trace.h" +#include "xfs_trans.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" +#include "xfs_da_btree.h" +#include "xfs_attr_sf.h" +#include "xfs_bmap.h" +#include "xfs_defer.h" +#include "xfs_log.h" +#include "xfs_xattr.h" +#include "xfs_parent.h" + +/* +* Parent pointer attribute handling. +* +* Because the attribute value is a filename component, it will never be longer
fs/xfs/libxfs/xfs_parent.c line 42
Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c new file +10
-0+* than 255 bytes. This means the attribute will always be a local format +* attribute as it is xfs_attr_leaf_entsize_local_max() for v5 filesystems will +* always be larger than this (max is 75% of block size). +* +* Creating a new parent attribute will always create a new attribute - there +* should never, ever be an existing attribute in the tree for a new inode. +* ENOSPC behavior is problematic - creating the inode without the parent +* pointer is effectively a corruption, so we allow parent attribute creation +* to dip into the reserve block pool to avoid unexpected ENOSPC errors from +* occurring.
Shouldn't we increase XFS_LINK_SPACE_RES to avoid this? The reserve pool isn't terribly large (8192 blocks) and was really only supposed to save us from an ENOSPC shutdown if an unwritten extent conversion in the writeback endio handler needs a few more blocks.
IOWs, we really ought to ENOSPC at transaction reservation time instead of draining the reserve pool.
Quoted 33 lines
fs/xfs/libxfs/xfs_parent.c new file +33
-0+*/ + + +/* Initializes a xfs_parent_name_rec to be stored as an attribute name */ +void +xfs_init_parent_name_rec ( +struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset) +{ +xfs_ino_t p_ino = ip->i_ino; +uint32_t p_gen =VFS_I (ip)->i_generation; + + rec->p_ino =cpu_to_be64 (p_ino); + rec->p_gen =cpu_to_be32 (p_gen); + rec->p_diroffset =cpu_to_be32 (p_diroffset); +} + +/* Initializes a xfs_parent_name_irec from an xfs_parent_name_rec */ +void +xfs_init_parent_name_irec ( +struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec) +{ + irec->p_ino =be64_to_cpu (rec->p_ino); + irec->p_gen =be32_to_cpu (rec->p_gen); + irec->p_diroffset =be32_to_cpu (rec->p_diroffset); +} + +int +xfs_parent_init( + xfs_mount_t *mp, + xfs_inode_t *ip,
fs/xfs/libxfs/xfs_parent.c line 85
Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c new file +10
-0+struct xfs_name *target_name, + struct xfs_parent_defer **parentp) +{ + struct xfs_parent_defer *parent; +int error; + +if (!xfs_has_parent (mp)) +return 0 ; + + error =xfs_attr_grab_log_assist (mp);
At some point we might want to consider boosting performance by setting XFS_SB_FEAT_INCOMPAT_LOG_XATTRS permanently when parent pointers are turned on, since adding the feature requires a synchronous bwrite of the primary superblock.
I /think/ this could be accomplished by setting the feature bit in mkfs and teaching xlog_clear_incompat to exit if xfs_has_parent()==true. Then we can skip the xfs_attr_grab_log_assist calls.
But, let's focus on getting this patchset into good enough shape that we can be confident that we don't need any ondisk format changes, and worry about speed later.
fs/xfs/libxfs/xfs_parent.c line 89
Quoted 4 lines
fs/xfs/libxfs/xfs_parent.c new file +4
-0+if (error) +return error; + + parent =kzalloc (sizeof (*parent),GFP_KERNEL );
These objects are going to be created and freed fairly frequently; could you please convert these to a kmem cache? (That can be a cleanup at the end.)
Quoted 189 lines
fs/xfs/libxfs/xfs_parent.c new file +45
-0+if (!parent) +return -ENOMEM ; + +/* init parent da_args */ + parent->args.dp = ip; + parent->args.geo = mp->m_attr_geo; + parent->args.whichfork =XFS_ATTR_FORK ; + parent->args.attr_filter =XFS_ATTR_PARENT ; + parent->args.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ; + parent->args.name = (const uint8_t *)&parent->rec; + parent->args.namelen =sizeof (struct xfs_parent_name_rec ); + +if (target_name) { + parent->args.value = (void *)target_name->name; + parent->args.valuelen = target_name->len; + } + + *parentp = parent; +return 0 ; +} + +int +xfs_parent_defer_add ( +struct xfs_trans *tp, +struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, ip, diroffset); + args->trans = tp; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_add (args); +} + +void +xfs_parent_cancel ( +xfs_mount_t *mp, +struct xfs_parent_defer *parent) +{ +xlog_drop_incompat_feat (mp->m_log); +kfree (parent); +} +fs/xfs/libxfs/xfs_parent.h new file +34
-0@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* +* Dynamically allocd structure used to wrap the needed data to pass around +* the defer ops machinery +*/ +struct xfs_parent_defer { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +/* +* Parent pointer attribute prototypes +*/ +void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset); +void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec); +int xfs_parent_init (xfs_mount_t *mp,xfs_inode_t *ip, +struct xfs_name *target_name, +struct xfs_parent_defer **parentp); +int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset); +void xfs_parent_cancel (xfs_mount_t *mp,struct xfs_parent_defer *parent); + +#endif /* __XFS_PARENT_H__ */ fs/xfs/xfs_inode.c +28
-5@@ -37,6 +37,8 @@#include "xfs_reflink.h" #include "xfs_ag.h" #include "xfs_log_priv.h" +#include "xfs_parent.h" +#include "xfs_xattr.h" struct kmem_cache *xfs_inode_cache;@@ -950,7 +952,7 @@ xfs_bumplink( int xfs_create( struct user_namespace *mnt_userns, - xfs_inode_t *dp, + struct xfs_inode *dp, struct xfs_name *name, umode_t mode, dev_t rdev,@@ -962,7 +964,7 @@ xfs_create(struct xfs_inode *ip =NULL ;struct xfs_trans *tp =NULL ;int error; -bool unlock_dp_on_error = false; +bool unlock_dp_on_error = false;prid_t prid;struct xfs_dquot *udqp =NULL ;struct xfs_dquot *gdqp =NULL ;@@ -970,6 +972,8 @@ xfs_create(struct xfs_trans_res *tres;uint resblks;xfs_ino_t ino; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_create (dp, name);@@ -996,6 +1000,12 @@ xfs_create( tres = &M_RES (mp)->tr_create; } +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, dp, name, &parent); +if (error) + goto out_release_dquots; + } +/* * Initially assume that the file does not exist and * reserve the resources for that case. If that is not @@ -1011,7 +1021,7 @@ xfs_create( resblks, &tp); }if (error) - goto out_release_dquots; + goto drop_incompat;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;@@ -1021,6 +1031,7 @@ xfs_create(* entry pointing to them, but a directory also the "." entry * pointing to itself. */ + init_xattrs |=xfs_has_parent (mp); error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error = xfs_init_new_inode(mnt_userns, tp, dp, ino, mode,@@ -1035,11 +1046,12 @@ xfs_create(* the transaction cancel unlocking dp so don't do it explicitly in the * error path. */ -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); unlock_dp_on_error = false; error =xfs_dir_createname (tp, dp, name, ip->i_ino, - resblks -XFS_IALLOC_SPACE_RES (mp),NULL ); + resblks -XFS_IALLOC_SPACE_RES (mp), + &diroffset);if (error) {ASSERT (error != -ENOSPC ); goto out_trans_cancel;@@ -1055,6 +1067,17 @@ xfs_create(xfs_bumplink (tp, dp); } +/* +* If we have parent pointers, we need to add the attribute containing +* the parent information now. +*/ +if (parent) { + parent->args.dp = ip; + error =xfs_parent_defer_add (tp, dp, parent, diroffset); +if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * create transaction goes to disk before returning to
fs/xfs/xfs_inode.c xfs_create( · line 1106
Quoted 4 lines
fs/xfs/xfs_inode.c +1
-0@@ -1080,6 +1103,7 @@ xfs_create( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT );
I don't think we need the ILOCK class annotations for unlocks.
Other than the two things I asked about, this is looking good.
--D
Quoted 41 lines
fs/xfs/xfs_inode.c +3
-0return 0 ; out_trans_cancel:@@ -1094,6 +1118,9 @@ xfs_create(xfs_finish_inode_setup (ip);xfs_irele (ip); } + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);fs/xfs/xfs_xattr.c +1
-1@@ -27,7 +27,7 @@* they must release the permission by calling xlog_drop_incompat_feat * when they're done. */ -static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {fs/xfs/xfs_xattr.h +1
-0@@ -7,6 +7,7 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *xfs_xattr_handlers[];Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 12/18] xfs: parent pointer attribute creation
Quoted 164 lines · Darrick J. Wong
Quoted 5 lines · Allison Henderson
Add parent pointer attribute during
xfs_create, and subroutines to initialize attributes[bfoster: rebase, use VFS inode generation] [achender: rebased, changed
__unint32_ttoxfs_dir2_dataptr_t,Nit:
uint32_t, notunint32_t.Quoted 111 lines
fixed some null pointer bugs,
merged error handling patch,
remove unnecessary ENOSPC handling inxfs_attr_set_first_parent]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_attr.h | 4 +- fs/xfs/libxfs/xfs_parent.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 34 ++++++++++ fs/xfs/xfs_inode.c | 37 ++++++++-- fs/xfs/xfs_xattr.c | 2 +- fs/xfs/xfs_xattr.h | 1 + 8 files changed, 208 insertions(+), 9 deletions(-)fs/xfs/Makefile +1
-0@@ -40,6 +40,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_fork.o \ xfs_inode_buf.o \ xfs_log_rlimit.o \ + xfs_parent.o \ xfs_ag_resv.o \ xfs_rmap.o \ xfs_rmap_btree.o \fs/xfs/libxfs/xfs_attr.c +2
-2@@ -880,7 +880,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set or remove) */@@ -898,7 +898,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {fs/xfs/libxfs/xfs_attr.h +3
-1@@ -544,6 +544,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);@@ -552,7 +553,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/* * Check to see if the attr should be upgraded from non-existent or shortform to * single-leaf-block attribute list. fs/xfs/libxfs/xfs_parent.c new file +32
-0@@ -0,0 +1,134 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All rights reserved. +*/ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_da_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_bmap_btree.h" +#include "xfs_inode.h" +#include "xfs_error.h" +#include "xfs_trace.h" +#include "xfs_trans.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" +#include "xfs_da_btree.h" +#include "xfs_attr_sf.h" +#include "xfs_bmap.h" +#include "xfs_defer.h" +#include "xfs_log.h" +#include "xfs_xattr.h" +#include "xfs_parent.h" + +/* +* Parent pointer attribute handling. +* +* Because the attribute value is a filename component, it will never be longer fs/xfs/libxfs/xfs_parent.c line 42
Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c new file +10
-0+* than 255 bytes. This means the attribute will always be a local format +* attribute as it is xfs_attr_leaf_entsize_local_max() for v5 filesystems will +* always be larger than this (max is 75% of block size). +* +* Creating a new parent attribute will always create a new attribute - there +* should never, ever be an existing attribute in the tree for a new inode. +* ENOSPC behavior is problematic - creating the inode without the parent +* pointer is effectively a corruption, so we allow parent attribute creation +* to dip into the reserve block pool to avoid unexpected ENOSPC errors from +* occurring. Shouldn't we increase
XFS_LINK_SPACE_RESto avoid this? The reserve pool isn't terribly large (8192 blocks) and was really only supposed to save us from an ENOSPC shutdown if an unwritten extent conversion in the writeback endio handler needs a few more blocks.IOWs, we really ought to ENOSPC at transaction reservation time instead of draining the reserve pool.
Quoted 25 lines
fs/xfs/libxfs/xfs_parent.c new file +25
-0+*/ + + +/* Initializes a xfs_parent_name_rec to be stored as an attribute name */ +void +xfs_init_parent_name_rec ( +struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset) +{ +xfs_ino_t p_ino = ip->i_ino; +uint32_t p_gen =VFS_I (ip)->i_generation; + + rec->p_ino =cpu_to_be64 (p_ino); + rec->p_gen =cpu_to_be32 (p_gen); + rec->p_diroffset =cpu_to_be32 (p_diroffset); +} + +/* Initializes a xfs_parent_name_irec from an xfs_parent_name_rec */ +void +xfs_init_parent_name_irec ( +struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec) +{ + irec->p_ino =be64_to_cpu (rec->p_ino);
Quoted 8 lines
fs/xfs/libxfs/xfs_parent.c line 75
Quoted 8 lines
fs/xfs/libxfs/xfs_parent.c new file +8
-0+ irec->p_gen =be32_to_cpu (rec->p_gen); + irec->p_diroffset =be32_to_cpu (rec->p_diroffset); +} + +int +xfs_parent_init( + xfs_mount_t *mp, + xfs_inode_t *ip,
More nits: Please don't use struct typedefs here.
Quoted 207 lines
fs/xfs/libxfs/xfs_parent.c line 85
Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c new file +10
-0+struct xfs_name *target_name, + struct xfs_parent_defer **parentp) +{ + struct xfs_parent_defer *parent; +int error; + +if (!xfs_has_parent (mp)) +return 0 ; + + error =xfs_attr_grab_log_assist (mp);At some point we might want to consider boosting performance by setting
XFS_SB_FEAT_INCOMPAT_LOG_XATTRSpermanently when parent pointers are turned on, since adding the feature requires a synchronous bwrite of the primary superblock.I /think/ this could be accomplished by setting the feature bit in mkfs and teaching
xlog_clear_incompatto exit if xfs_has_parent()==true. Then we can skip thexfs_attr_grab_log_assistcalls.But, let's focus on getting this patchset into good enough shape that we can be confident that we don't need any ondisk format changes, and worry about speed later.
fs/xfs/libxfs/xfs_parent.c line 89
Quoted 4 lines
fs/xfs/libxfs/xfs_parent.c new file +4
-0+if (error) +return error; + + parent =kzalloc (sizeof (*parent),GFP_KERNEL );These objects are going to be created and freed fairly frequently; could you please convert these to a kmem cache? (That can be a cleanup at the end.)
Quoted 174 lines
fs/xfs/libxfs/xfs_parent.c new file +45
-0+if (!parent) +return -ENOMEM ; + +/* init parent da_args */ + parent->args.dp = ip; + parent->args.geo = mp->m_attr_geo; + parent->args.whichfork =XFS_ATTR_FORK ; + parent->args.attr_filter =XFS_ATTR_PARENT ; + parent->args.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ; + parent->args.name = (const uint8_t *)&parent->rec; + parent->args.namelen =sizeof (struct xfs_parent_name_rec ); + +if (target_name) { + parent->args.value = (void *)target_name->name; + parent->args.valuelen = target_name->len; + } + + *parentp = parent; +return 0 ; +} + +int +xfs_parent_defer_add ( +struct xfs_trans *tp, +struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, ip, diroffset); + args->trans = tp; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_add (args); +} + +void +xfs_parent_cancel ( +xfs_mount_t *mp, +struct xfs_parent_defer *parent) +{ +xlog_drop_incompat_feat (mp->m_log); +kfree (parent); +} +fs/xfs/libxfs/xfs_parent.h new file +34
-0@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* +* Dynamically allocd structure used to wrap the needed data to pass around +* the defer ops machinery +*/ +struct xfs_parent_defer { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +/* +* Parent pointer attribute prototypes +*/ +void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset); +void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec); +int xfs_parent_init (xfs_mount_t *mp,xfs_inode_t *ip, +struct xfs_name *target_name, +struct xfs_parent_defer **parentp); +int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset); +void xfs_parent_cancel (xfs_mount_t *mp,struct xfs_parent_defer *parent); + +#endif /* __XFS_PARENT_H__ */ fs/xfs/xfs_inode.c +17
-5@@ -37,6 +37,8 @@#include "xfs_reflink.h" #include "xfs_ag.h" #include "xfs_log_priv.h" +#include "xfs_parent.h" +#include "xfs_xattr.h" struct kmem_cache *xfs_inode_cache;@@ -950,7 +952,7 @@ xfs_bumplink( int xfs_create( struct user_namespace *mnt_userns, - xfs_inode_t *dp, + struct xfs_inode *dp, struct xfs_name *name, umode_t mode, dev_t rdev,@@ -962,7 +964,7 @@ xfs_create(struct xfs_inode *ip =NULL ;struct xfs_trans *tp =NULL ;int error; -bool unlock_dp_on_error = false; +bool unlock_dp_on_error = false;prid_t prid;struct xfs_dquot *udqp =NULL ;struct xfs_dquot *gdqp =NULL ;@@ -970,6 +972,8 @@ xfs_create(struct xfs_trans_res *tres;uint resblks;xfs_ino_t ino; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_create (dp, name);@@ -996,6 +1000,12 @@ xfs_create( tres = &M_RES (mp)->tr_create; } +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, dp, name, &parent); +if (error) + goto out_release_dquots; + } +/* * Initially assume that the file does not exist and * reserve the resources for that case. If that is not @@ -1011,7 +1021,7 @@ xfs_create( resblks, &tp); }if (error) - goto out_release_dquots; + goto drop_incompat;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;@@ -1021,6 +1031,7 @@ xfs_create(* entry pointing to them, but a directory also the "." entry * pointing to itself. */ + init_xattrs |=xfs_has_parent (mp); error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error = xfs_init_new_inode(mnt_userns, tp, dp, ino, mode,@@ -1035,11 +1046,12 @@ xfs_create(* the transaction cancel unlocking dp so don't do it explicitly in the * error path. */ -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); unlock_dp_on_error = false; error =xfs_dir_createname (tp, dp, name, ip->i_ino, - resblks -XFS_IALLOC_SPACE_RES (mp),NULL ); + resblks -XFS_IALLOC_SPACE_RES (mp), + &diroffset);if (error) {ASSERT (error != -ENOSPC ); goto out_trans_cancel;@@ -1055,6 +1067,17 @@ xfs_create(xfs_bumplink (tp, dp); }
Quoted 6 lines
fs/xfs/xfs_inode.c xfs_create( · line 1075
Quoted 6 lines
fs/xfs/xfs_inode.c +6
-0+/* +* If we have parent pointers, we need to add the attribute containing +* the parent information now. +*/ +if (parent) { + parent->args.dp = ip;
...and on second thought, it seems a little odd that you pass @dp to xfs_parent_init only to override parent->args.dp here. Given that this doesn't do anything with @parent until here, why not pass NULL to the init function above?
--D
Quoted 61 lines
fs/xfs/xfs_inode.c xfs_create( · line 1106
Quoted 13 lines
fs/xfs/xfs_inode.c +6
-0+ error =xfs_parent_defer_add (tp, dp, parent, diroffset); +if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * create transaction goes to disk before returning to @@ -1080,6 +1103,7 @@ xfs_create( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT );I don't think we need the ILOCK class annotations for unlocks.
Other than the two things I asked about, this is looking good.
--D
Quoted 41 lines
fs/xfs/xfs_inode.c +3
-0return 0 ; out_trans_cancel:@@ -1094,6 +1118,9 @@ xfs_create(xfs_finish_inode_setup (ip);xfs_irele (ip); } + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);fs/xfs/xfs_xattr.c +1
-1@@ -27,7 +27,7 @@* they must release the permission by calling xlog_drop_incompat_feat * when they're done. */ -static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {fs/xfs/xfs_xattr.h +1
-0@@ -7,6 +7,7 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *xfs_xattr_handlers[];Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 12/18] xfs: parent pointer attribute creation
Quoted 191 lines · Darrick J. Wong
Quoted 190 lines · Darrick J. Wong
Quoted 6 lines · Allison Henderson
Add parent pointer attribute during
xfs_create, and subroutines to
initialize attributes[bfoster: rebase, use VFS inode generation] [achender: rebased, changed
__unint32_ttoxfs_dir2_dataptr_t,Nit:
uint32_t, notunint32_t.Quoted 130 lines
fixed some null pointer bugs,
merged error handling patch,
remove unnecessary ENOSPC handling inxfs_attr_set_first_parent]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_attr.h | 4 +- fs/xfs/libxfs/xfs_parent.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 34 ++++++++++ fs/xfs/xfs_inode.c | 37 ++++++++-- fs/xfs/xfs_xattr.c | 2 +- fs/xfs/xfs_xattr.h | 1 + 8 files changed, 208 insertions(+), 9 deletions(-)fs/xfs/Makefile +1
-0@@ -40,6 +40,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_fork.o \ xfs_inode_buf.o \ xfs_log_rlimit.o \ + xfs_parent.o \ xfs_ag_resv.o \ xfs_rmap.o \fs/xfs/Makefile +0
-0xfs_rmap_btree.o \fs/xfs/libxfs/xfs_attr.c +1
-1@@ -880,7 +880,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set orremove) */
fs/xfs/libxfs/xfs_attr.c +1
-1@@ -898,7 +898,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {fs/xfs/libxfs/xfs_attr.h +3
-1@@ -544,6 +544,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);@@ -552,7 +553,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length, int xfs_attr_calc_size(struct xfs_da_args *args, int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/*
* Check to see if the attr should be upgraded from non-existent
or shortform to
* single-leaf-block attribute list.fs/xfs/libxfs/xfs_parent.c new file +32
-0@@ -0,0 +1,134 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All rights reserved. +*/ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_da_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_bmap_btree.h" +#include "xfs_inode.h" +#include "xfs_error.h" +#include "xfs_trace.h" +#include "xfs_trans.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" +#include "xfs_da_btree.h" +#include "xfs_attr_sf.h" +#include "xfs_bmap.h" +#include "xfs_defer.h" +#include "xfs_log.h" +#include "xfs_xattr.h" +#include "xfs_parent.h" + +/* +* Parent pointer attribute handling. +* +* Because the attribute value is a filename component, it will never be longer
+ * than 255 bytes. This means the attribute will always be a local format
+ * attribute as it isxfs_attr_leaf_entsize_local_max()for v5 filesystems will
+ * always be larger than this (max is 75% of block size).
+ *
+ * Creating a new parent attribute will always create a new attribute - there
+ * should never, ever be an existing attribute in the tree for aQuoted 8 lines
new inode.
+ * ENOSPC behavior is problematic - creating the inode without the parent
+ * pointer is effectively a corruption, so we allow parent attribute creation
+ * to dip into the reserve block pool to avoid unexpected ENOSPC errors from
+ * occurring.Shouldn't we increase
XFS_LINK_SPACE_RESto avoid this? The reserve
pool isn't terribly large (8192 blocks) and was really only supposed to
save us from an ENOSPC shutdown if an unwritten extent conversion in the
writeback endio handler needs a few more blocks.IOWs, we really ought to ENOSPC at transaction reservation time instead
of draining the reserve pool.Quoted 29 lines
+ */
fs/xfs/libxfs/xfs_parent.c new file +3
-0+ + +/* Initializes a xfs_parent_name_rec to be stored as an attribute name */
fs/xfs/libxfs/xfs_parent.c new file +15
-0+void +xfs_init_parent_name_rec ( +struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset) +{ +xfs_ino_t p_ino = ip->i_ino; +uint32_t p_gen =VFS_I (ip)->i_generation; + + rec->p_ino =cpu_to_be64 (p_ino); + rec->p_gen =cpu_to_be32 (p_gen); + rec->p_diroffset =cpu_to_be32 (p_diroffset); +} + +/* Initializes a xfs_parent_name_irec from an
xfs_parent_name_rec*/fs/xfs/libxfs/xfs_parent.c new file +8
-0+void +xfs_init_parent_name_irec ( +struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec) +{ + irec->p_ino =be64_to_cpu (rec->p_ino); + irec->p_gen =be32_to_cpu (rec->p_gen); + irec->p_diroffset =be32_to_cpu (rec->p_diroffset);
Quoted 8 lines
Quoted 6 lines
fs/xfs/libxfs/xfs_parent.c line 75
Quoted 6 lines
fs/xfs/libxfs/xfs_parent.c new file +6
-0+} + +int +xfs_parent_init( + xfs_mount_t *mp, + xfs_inode_t *ip,More nits: Please don't use struct typedefs here.
Sure, will fix
Quoted 224 lines
Quoted 224 lines
fs/xfs/libxfs/xfs_parent.c line 85
Quoted 11 lines
fs/xfs/libxfs/xfs_parent.c new file +10
-0+struct xfs_name *target_name, + struct xfs_parent_defer **parentp) +{ + struct xfs_parent_defer *parent; +int error; + +if (!xfs_has_parent (mp)) +return 0 ; + + error =xfs_attr_grab_log_assist (mp);At some point we might want to consider boosting performance by setting
XFS_SB_FEAT_INCOMPAT_LOG_XATTRSpermanently when parent pointers are
turned on, since adding the feature requires a synchronous bwrite of the
primary superblock.I /think/ this could be accomplished by setting the feature bit in mkfs
and teachingxlog_clear_incompatto exit if xfs_has_parent()==true. Then we can skip thexfs_attr_grab_log_assistcalls.But, let's focus on getting this patchset into good enough shape that
we can be confident that we don't need any ondisk format changes, and
worry about speed later.fs/xfs/libxfs/xfs_parent.c line 89
Quoted 4 lines
fs/xfs/libxfs/xfs_parent.c new file +4
-0+if (error) +return error; + + parent =kzalloc (sizeof (*parent),GFP_KERNEL );These objects are going to be created and freed fairly frequently; could
you please convert these to a kmem cache? (That can be a cleanup at the
end.)Quoted 182 lines
fs/xfs/libxfs/xfs_parent.c new file +45
-0+if (!parent) +return -ENOMEM ; + +/* init parent da_args */ + parent->args.dp = ip; + parent->args.geo = mp->m_attr_geo; + parent->args.whichfork =XFS_ATTR_FORK ; + parent->args.attr_filter =XFS_ATTR_PARENT ; + parent->args.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ; + parent->args.name = (const uint8_t *)&parent->rec; + parent->args.namelen =sizeof (struct xfs_parent_name_rec ); + +if (target_name) { + parent->args.value = (void *)target_name->name; + parent->args.valuelen = target_name->len; + } + + *parentp = parent; +return 0 ; +} + +int +xfs_parent_defer_add ( +struct xfs_trans *tp, +struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, ip, diroffset); + args->trans = tp; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_add (args); +} + +void +xfs_parent_cancel ( +xfs_mount_t *mp, +struct xfs_parent_defer *parent) +{ +xlog_drop_incompat_feat (mp->m_log); +kfree (parent); +} +fs/xfs/libxfs/xfs_parent.h new file +10
-0@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* +* Dynamically allocd structure used to wrap the needed data to pass around
+ * the defer ops machinery
+ */fs/xfs/libxfs/xfs_parent.h new file +12
-0+struct xfs_parent_defer { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +/* +* Parent pointer attribute prototypes +*/ +void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset); +void xfs_init_parent_name_irec(struct xfs_parent_name_irec*irec,
fs/xfs/libxfs/xfs_parent.h new file +5
-0+ struct xfs_parent_name_rec *rec); +int xfs_parent_init (xfs_mount_t *mp,xfs_inode_t *ip, +struct xfs_name *target_name, +struct xfs_parent_defer **parentp); +int xfs_parent_defer_add(struct xfs_trans *tp, struct xfs_inode*ip,
fs/xfs/libxfs/xfs_parent.h new file +3
-0+struct xfs_parent_defer *parent, + xfs_dir2_dataptr_t diroffset); +void xfs_parent_cancel(xfs_mount_t *mp, struct xfs_parent_defer*parent);
fs/xfs/libxfs/xfs_parent.h new file +2
-0+ +#endif /* __XFS_PARENT_H__ */ fs/xfs/xfs_inode.c +14
-3@@ -37,6 +37,8 @@#include "xfs_reflink.h" #include "xfs_ag.h" #include "xfs_log_priv.h" +#include "xfs_parent.h" +#include "xfs_xattr.h" struct kmem_cache *xfs_inode_cache;@@ -950,7 +952,7 @@ xfs_bumplink( int xfs_create( struct user_namespace *mnt_userns, - xfs_inode_t *dp, + struct xfs_inode *dp, struct xfs_name *name, umode_t mode, dev_t rdev,@@ -962,7 +964,7 @@ xfs_create(struct xfs_inode *ip =NULL ;struct xfs_trans *tp =NULL ;int error; -bool unlock_dp_on_error = false; +bool unlock_dp_on_error = false;prid_t prid;struct xfs_dquot *udqp =NULL ;struct xfs_dquot *gdqp =NULL ;@@ -970,6 +972,8 @@ xfs_create(struct xfs_trans_res *tres;uint resblks;xfs_ino_t ino; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_create (dp, name);@@ -996,6 +1000,12 @@ xfs_create( tres = &M_RES (mp)->tr_create; } +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, dp, name, &parent); +if (error) + goto out_release_dquots; + } +/* * Initially assume that the file does not exist and * reserve the resources for that case. If that is not @@ -1011,7 +1021,7 @@ xfs_create( resblks, &tp); }if (error) - goto out_release_dquots; + goto drop_incompat;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;@@ -1021,6 +1031,7 @@ xfs_create(* entry pointing to them, but a directory also the "." entry * pointing to itself. */ + init_xattrs |=xfs_has_parent (mp); error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error = xfs_init_new_inode(mnt_userns, tp, dp, ino,mode,
fs/xfs/xfs_inode.c +0
-0@@ -1035,11 +1046,12 @@ xfs_create(* the transaction cancel unlocking dp so don't do it explicitly in the
fs/xfs/xfs_inode.c +1
-2* error path. */ -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); unlock_dp_on_error = false; error = xfs_dir_createname(tp, dp, name, ip->i_ino, - resblks - XFS_IALLOC_SPACE_RES(mp),NULL);
fs/xfs/xfs_inode.c +2
-0+ resblks -XFS_IALLOC_SPACE_RES (mp), + &diroffset);if (error) {ASSERT (error != -ENOSPC ); goto out_trans_cancel;@@ -1055,6 +1067,17 @@ xfs_create(xfs_bumplink (tp, dp); }
Quoted 13 lines
Quoted 7 lines
fs/xfs/xfs_inode.c xfs_create( · line 1075
Quoted 7 lines
fs/xfs/xfs_inode.c +2
-0+/* +* If we have parent pointers, we need to add the attribute containing
fs/xfs/xfs_inode.c +4
-0+* the parent information now. +*/ +if (parent) { + parent->args.dp = ip;...and on second thought, it seems a little odd that you pass @dp to
xfs_parent_initonly to overrideparent->args.dphere. Given that this
doesn't do anything with @parent until here, why not pass NULL to the init function above?
Sure, the init helpers are helpful, but do create some out of order initializing since sometimes the required parameters are not all available at the same time. Will update.
Thanks!
Allison
Quoted 66 lines
--D
Quoted 63 lines
Quoted 10 lines
fs/xfs/xfs_inode.c +1
-0+ error = xfs_parent_defer_add(tp, dp, parent,diroffset);
fs/xfs/xfs_inode.c +4
-0+if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * create transaction goes to disk before returning to fs/xfs/xfs_inode.c xfs_create( · line 1106
Quoted 4 lines
fs/xfs/xfs_inode.c +1
-0@@ -1080,6 +1103,7 @@ xfs_create( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT );I don't think we need the ILOCK class annotations for unlocks.
Other than the two things I asked about, this is looking good.
--D
Quoted 42 lines
fs/xfs/xfs_inode.c +3
-0return 0 ; out_trans_cancel:@@ -1094,6 +1118,9 @@ xfs_create(xfs_finish_inode_setup (ip);xfs_irele (ip); } + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);fs/xfs/xfs_xattr.c +0
-0@@ -27,7 +27,7 @@* they must release the permission by calling
xlog_drop_incompat_feat
* when they're done.
*/fs/xfs/xfs_xattr.c +1
-1-static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {fs/xfs/xfs_xattr.h +1
-0@@ -7,6 +7,7 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *xfs_xattr_handlers[];Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 12/18] xfs: parent pointer attribute creation
In reply to Darrick J. WongQuoted 8 lines · Darrick J. Wong
Quoted 5 lines · Allison Henderson
Add parent pointer attribute during
xfs_create, and subroutines to initialize attributes[bfoster: rebase, use VFS inode generation] [achender: rebased, changed
__unint32_ttoxfs_dir2_dataptr_t,Nit:
uint32_t, notunint32_t.
I actually thought about removing this little change log all together?
I had initially added that to follow suit with Brians style, but
really the set has undergone so many updates, trying to keep a log here seems a bit silly. Unless there's a reason people would like to hang on to them, I think maybe we should just clean them out?
Quoted 138 lines
Quoted 138 lines
fixed some null pointer bugs,
merged error handling patch,
remove unnecessary ENOSPC handling inxfs_attr_set_first_parent]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_attr.h | 4 +- fs/xfs/libxfs/xfs_parent.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 34 ++++++++++ fs/xfs/xfs_inode.c | 37 ++++++++-- fs/xfs/xfs_xattr.c | 2 +- fs/xfs/xfs_xattr.h | 1 + 8 files changed, 208 insertions(+), 9 deletions(-)fs/xfs/Makefile +1
-0@@ -40,6 +40,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_fork.o \ xfs_inode_buf.o \ xfs_log_rlimit.o \ + xfs_parent.o \ xfs_ag_resv.o \ xfs_rmap.o \fs/xfs/Makefile +0
-0xfs_rmap_btree.o \fs/xfs/libxfs/xfs_attr.c +1
-1@@ -880,7 +880,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set orremove) */
fs/xfs/libxfs/xfs_attr.c +1
-1@@ -898,7 +898,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {fs/xfs/libxfs/xfs_attr.h +3
-1@@ -544,6 +544,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);@@ -552,7 +553,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length, int xfs_attr_calc_size(struct xfs_da_args *args, int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/*
* Check to see if the attr should be upgraded from non-existent
or shortform to
* single-leaf-block attribute list.fs/xfs/libxfs/xfs_parent.c new file +32
-0@@ -0,0 +1,134 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All rights reserved. +*/ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_da_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_bmap_btree.h" +#include "xfs_inode.h" +#include "xfs_error.h" +#include "xfs_trace.h" +#include "xfs_trans.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" +#include "xfs_da_btree.h" +#include "xfs_attr_sf.h" +#include "xfs_bmap.h" +#include "xfs_defer.h" +#include "xfs_log.h" +#include "xfs_xattr.h" +#include "xfs_parent.h" + +/* +* Parent pointer attribute handling. +* +* Because the attribute value is a filename component, it will never be longer
+ * than 255 bytes. This means the attribute will always be a local format
+ * attribute as it isxfs_attr_leaf_entsize_local_max()for v5 filesystems will
+ * always be larger than this (max is 75% of block size).
+ *
+ * Creating a new parent attribute will always create a new attribute - there
+ * should never, ever be an existing attribute in the tree for a new inode.
+ * ENOSPC behavior is problematic - creating the inode without the parent
+ * pointer is effectively a corruption, so we allow parent attribute creation
+ * to dip into the reserve block pool to avoid unexpected ENOSPC errors from
Quoted 9 lines
Quoted 1 line
+ * occurring.
Shouldn't we increase
XFS_LINK_SPACE_RESto avoid this? The reserve pool isn't terribly large (8192 blocks) and was really only supposed to
save us from an ENOSPC shutdown if an unwritten extent conversion in the
writeback endio handler needs a few more blocks.
Did you maybe mean XFS_IALLOC_SPACE_RES? That looks like the macro that's getting used below in xfs_create
Quoted 3 lines
IOWs, we really ought to ENOSPC at transaction reservation time instead
of draining the reserve pool.
It looks like we do that in most cases. I dont actually see rsvd getting set, other than in xfs_attr_set. Which isnt used in parent pointer updating, and should probably be removed. I suspect it's a relic of the pre-larp version of the set. So perhaps the comment is stale and should be removed as well.
Quoted 54 lines
Quoted 36 lines
+ */
fs/xfs/libxfs/xfs_parent.c new file +3
-0+ + +/* Initializes a xfs_parent_name_rec to be stored as an attribute name */
fs/xfs/libxfs/xfs_parent.c new file +15
-0+void +xfs_init_parent_name_rec ( +struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset) +{ +xfs_ino_t p_ino = ip->i_ino; +uint32_t p_gen =VFS_I (ip)->i_generation; + + rec->p_ino =cpu_to_be64 (p_ino); + rec->p_gen =cpu_to_be32 (p_gen); + rec->p_diroffset =cpu_to_be32 (p_diroffset); +} + +/* Initializes a xfs_parent_name_irec from an xfs_parent_name_rec */
fs/xfs/libxfs/xfs_parent.c new file +14
-0+void +xfs_init_parent_name_irec ( +struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec) +{ + irec->p_ino =be64_to_cpu (rec->p_ino); + irec->p_gen =be32_to_cpu (rec->p_gen); + irec->p_diroffset =be32_to_cpu (rec->p_diroffset); +} + +int +xfs_parent_init( + xfs_mount_t *mp, + xfs_inode_t *ip,fs/xfs/libxfs/xfs_parent.c line 85
Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c new file +10
-0+struct xfs_name *target_name, + struct xfs_parent_defer **parentp) +{ + struct xfs_parent_defer *parent; +int error; + +if (!xfs_has_parent (mp)) +return 0 ; + + error =xfs_attr_grab_log_assist (mp);At some point we might want to consider boosting performance by setting
XFS_SB_FEAT_INCOMPAT_LOG_XATTRSpermanently when parent pointers are turned on, since adding the feature requires a synchronous bwrite of the
primary superblock.
Quoted 8 lines
I /think/ this could be accomplished by setting the feature bit in mkfs
and teachingxlog_clear_incompatto exit if xfs_has_parent()==true. Then we can skip thexfs_attr_grab_log_assistcalls.But, let's focus on getting this patchset into good enough shape that we can be confident that we don't need any ondisk format changes, and worry about speed later.
Yep, I will add that to the mkfs side. I do have the user space updates on git hub, but I dont want to patch bomb the list with it just yet because it's just too much to review all at once. It makes sense to get the kernel updates out of the way first.
Quoted 11 lines
fs/xfs/libxfs/xfs_parent.c line 89
Quoted 5 lines
fs/xfs/libxfs/xfs_parent.c new file +4
-0+if (error) +return error; + + parent =kzalloc (sizeof (*parent),GFP_KERNEL );These objects are going to be created and freed fairly frequently; could
you please convert these to a kmem cache? (That can be a cleanup at the
end.)
Sure, will do
Quoted 200 lines
Quoted 200 lines
fs/xfs/libxfs/xfs_parent.c new file +45
-0+if (!parent) +return -ENOMEM ; + +/* init parent da_args */ + parent->args.dp = ip; + parent->args.geo = mp->m_attr_geo; + parent->args.whichfork =XFS_ATTR_FORK ; + parent->args.attr_filter =XFS_ATTR_PARENT ; + parent->args.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ; + parent->args.name = (const uint8_t *)&parent->rec; + parent->args.namelen =sizeof (struct xfs_parent_name_rec ); + +if (target_name) { + parent->args.value = (void *)target_name->name; + parent->args.valuelen = target_name->len; + } + + *parentp = parent; +return 0 ; +} + +int +xfs_parent_defer_add ( +struct xfs_trans *tp, +struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, ip, diroffset); + args->trans = tp; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_add (args); +} + +void +xfs_parent_cancel ( +xfs_mount_t *mp, +struct xfs_parent_defer *parent) +{ +xlog_drop_incompat_feat (mp->m_log); +kfree (parent); +} +fs/xfs/libxfs/xfs_parent.h new file +10
-0@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* +* Dynamically allocd structure used to wrap the needed data to pass around
+ * the defer ops machinery
+ */fs/xfs/libxfs/xfs_parent.h new file +17
-0+struct xfs_parent_defer { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +/* +* Parent pointer attribute prototypes +*/ +void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, +struct xfs_inode *ip, +uint32_t p_diroffset); +void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec); +int xfs_parent_init (xfs_mount_t *mp,xfs_inode_t *ip, +struct xfs_name *target_name, +struct xfs_parent_defer **parentp); +int xfs_parent_defer_add(struct xfs_trans *tp, struct xfs_inode*ip,
fs/xfs/libxfs/xfs_parent.h new file +3
-0+struct xfs_parent_defer *parent, + xfs_dir2_dataptr_t diroffset); +void xfs_parent_cancel(xfs_mount_t *mp, struct xfs_parent_defer*parent);
fs/xfs/libxfs/xfs_parent.h new file +2
-0+ +#endif /* __XFS_PARENT_H__ */ fs/xfs/xfs_inode.c +14
-3@@ -37,6 +37,8 @@#include "xfs_reflink.h" #include "xfs_ag.h" #include "xfs_log_priv.h" +#include "xfs_parent.h" +#include "xfs_xattr.h" struct kmem_cache *xfs_inode_cache;@@ -950,7 +952,7 @@ xfs_bumplink( int xfs_create( struct user_namespace *mnt_userns, - xfs_inode_t *dp, + struct xfs_inode *dp, struct xfs_name *name, umode_t mode, dev_t rdev,@@ -962,7 +964,7 @@ xfs_create(struct xfs_inode *ip =NULL ;struct xfs_trans *tp =NULL ;int error; -bool unlock_dp_on_error = false; +bool unlock_dp_on_error = false;prid_t prid;struct xfs_dquot *udqp =NULL ;struct xfs_dquot *gdqp =NULL ;@@ -970,6 +972,8 @@ xfs_create(struct xfs_trans_res *tres;uint resblks;xfs_ino_t ino; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_create (dp, name);@@ -996,6 +1000,12 @@ xfs_create( tres = &M_RES (mp)->tr_create; } +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, dp, name, &parent); +if (error) + goto out_release_dquots; + } +/* * Initially assume that the file does not exist and * reserve the resources for that case. If that is not @@ -1011,7 +1021,7 @@ xfs_create( resblks, &tp); }if (error) - goto out_release_dquots; + goto drop_incompat;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;@@ -1021,6 +1031,7 @@ xfs_create(* entry pointing to them, but a directory also the "." entry * pointing to itself. */ + init_xattrs |=xfs_has_parent (mp); error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error = xfs_init_new_inode(mnt_userns, tp, dp, ino,mode,
fs/xfs/xfs_inode.c +0
-0@@ -1035,11 +1046,12 @@ xfs_create(* the transaction cancel unlocking dp so don't do it explicitly in the
fs/xfs/xfs_inode.c +1
-2* error path. */ -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); unlock_dp_on_error = false; error = xfs_dir_createname(tp, dp, name, ip->i_ino, - resblks - XFS_IALLOC_SPACE_RES(mp),NULL);
fs/xfs/xfs_inode.c +4
-0+ resblks -XFS_IALLOC_SPACE_RES (mp), + &diroffset);if (error) {ASSERT (error != -ENOSPC ); goto out_trans_cancel;@@ -1055,6 +1067,17 @@ xfs_create(xfs_bumplink (tp, dp); } +/* +* If we have parent pointers, we need to add the attribute containing
fs/xfs/xfs_inode.c +5
-0+* the parent information now. +*/ +if (parent) { + parent->args.dp = ip; + error = xfs_parent_defer_add(tp, dp, parent,diroffset);
fs/xfs/xfs_inode.c +4
-0+if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * create transaction goes to disk before returning to @@ -1080,6 +1103,7 @@ xfs_create(
Quoted 7 lines
fs/xfs/xfs_inode.c xfs_create( · line 1106
Quoted 3 lines
fs/xfs/xfs_inode.c +1
-0*ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT );I don't think we need the ILOCK class annotations for unlocks.
Other than the two things I asked about, this is looking good.
Ok, will remove the XFS_ILOCK_PARENT. Thanks for the reviews!
Allison
Quoted 45 lines
--D
Quoted 42 lines
fs/xfs/xfs_inode.c +3
-0return 0 ; out_trans_cancel:@@ -1094,6 +1118,9 @@ xfs_create(xfs_finish_inode_setup (ip);xfs_irele (ip); } + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);fs/xfs/xfs_xattr.c +0
-0@@ -27,7 +27,7 @@* they must release the permission by calling
xlog_drop_incompat_feat
* when they're done.
*/fs/xfs/xfs_xattr.c +1
-1-static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {fs/xfs/xfs_xattr.h +1
-0@@ -7,6 +7,7 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *xfs_xattr_handlers[];Signature · 1 line
2.25.1
PATCH RESEND v2 13/18 xfs: add parent attributes to link
In reply to Allison HendersonThis patch modifies xfs_link to add a parent pointer to the inode.
[bfoster: rebase, use VFS inode fields, fix xfs_bmap_finish() usage] [achender: rebased, changed __unint32_t to xfs_dir2_dataptr_t,
fixed null pointer bugs]
--- fs/xfs/xfs_inode.c | 43 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 9 deletions(-)
fs/xfs/xfs_inode.c +34 -9
@@ -1228,14 +1228,16 @@ xfs_create_tmpfile(int xfs_link ( -xfs_inode_t *tdp, -xfs_inode_t *sip, +struct xfs_inode *tdp, +struct xfs_inode *sip,struct xfs_name *target_name) { -xfs_mount_t *mp = tdp->i_mount; -xfs_trans_t *tp; +struct xfs_mount *mp = tdp->i_mount; +struct xfs_trans *tp;int error, nospace_error =0 ;int resblks; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_link (tdp, target_name);
@@ -1252,11 +1254,17 @@ xfs_link(if (error) goto std_return; +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, sip, target_name, &parent); +if (error) + goto std_return; + } + resblks =XFS_LINK_SPACE_RES (mp, target_name->len); error =xfs_trans_alloc_dir (tdp, &M_RES (mp)->tr_link, sip, &resblks, &tp, &nospace_error);if (error) - goto std_return; + goto drop_incompat;/* * If we are using project inheritance, we only allow hard link
@@ -1289,14 +1297,26 @@ xfs_link( } error =xfs_dir_createname (tp, tdp, target_name, sip->i_ino, - resblks,NULL ); + resblks, &diroffset);if (error) - goto error_return; + goto out_defer_cancel;xfs_trans_ichgtime (tp, tdp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, tdp,XFS_ILOG_CORE );xfs_bumplink (tp, sip); +/* +* If we have parent pointers, we now need to add the parent record to +* the attribute fork of the inode. If this is the initial parent +* attribute, we need to create it correctly, otherwise we can just add +* the parent to the inode. +*/ +if (parent) { + error =xfs_parent_defer_add (tp, tdp, parent, diroffset); +if (error) + goto out_defer_cancel; + } +/* * If this is a synchronous mount, make sure that the * link transaction goes to disk before returning to
@@ -1310,11 +1330,16 @@ xfs_link(xfs_iunlock (sip,XFS_ILOCK_EXCL );return error; - error_return: +out_defer_cancel: +xfs_defer_cancel (tp); +error_return:xfs_trans_cancel (tp);xfs_iunlock (tdp,XFS_ILOCK_EXCL );xfs_iunlock (sip,XFS_ILOCK_EXCL ); - std_return: +drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); +std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;return error;
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 13/18] xfs: add parent attributes to link
Quoted 38 lines · Allison Henderson
This patch modifies
xfs_linkto add a parent pointer to the inode.[bfoster: rebase, use VFS inode fields, fix
xfs_bmap_finish()usage] [achender: rebased, changed__unint32_ttoxfs_dir2_dataptr_t,
fixed null pointer bugs]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_inode.c | 43 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 9 deletions(-)fs/xfs/xfs_inode.c +6
-4@@ -1228,14 +1228,16 @@ xfs_create_tmpfile(int xfs_link ( -xfs_inode_t *tdp, -xfs_inode_t *sip, +struct xfs_inode *tdp, +struct xfs_inode *sip,struct xfs_name *target_name) { -xfs_mount_t *mp = tdp->i_mount; -xfs_trans_t *tp; +struct xfs_mount *mp = tdp->i_mount; +struct xfs_trans *tp;int error, nospace_error =0 ;int resblks; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_link (tdp, target_name);
fs/xfs/xfs_inode.c xfs_link( · line 1258
Quoted 5 lines
fs/xfs/xfs_inode.c +2
-0@@ -1252,11 +1254,17 @@ xfs_link(if (error) goto std_return; +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, sip, target_name, &parent);
Why does xfs_parent_init check xfs_has_parent if the callers already do that?
fs/xfs/xfs_inode.c xfs_link( · line 1263
Quoted 5 lines
fs/xfs/xfs_inode.c +4
-0+if (error) + goto std_return; + } + resblks =XFS_LINK_SPACE_RES (mp, target_name->len);
Same comment about increasing XFS_LINK_SPACE_RES to accomodate xattr expansion as I had for the last patch.
Quoted 20 lines
fs/xfs/xfs_inode.c +3
-3error =xfs_trans_alloc_dir (tdp, &M_RES (mp)->tr_link, sip, &resblks, &tp, &nospace_error);if (error) - goto std_return; + goto drop_incompat;/* * If we are using project inheritance, we only allow hard link @@ -1289,14 +1297,26 @@ xfs_link( } error =xfs_dir_createname (tp, tdp, target_name, sip->i_ino, - resblks,NULL ); + resblks, &diroffset);if (error) - goto error_return; + goto out_defer_cancel;xfs_trans_ichgtime (tp, tdp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, tdp,XFS_ILOG_CORE );
fs/xfs/xfs_inode.c xfs_link( · line 1315
Quoted 10 lines
fs/xfs/xfs_inode.c +8
-0xfs_bumplink (tp, sip); +/* +* If we have parent pointers, we now need to add the parent record to +* the attribute fork of the inode. If this is the initial parent +* attribute, we need to create it correctly, otherwise we can just add +* the parent to the inode. +*/ +if (parent) { + error =xfs_parent_defer_add (tp, tdp, parent, diroffset);
A followup to the comments I made to the previous patch about parent->args.dp --
Since you're partially initializing the xfs_defer_parent structure before you even have the dir offset, why not delay initializing the parent and child pointers until the xfs_parent_defer_add step?
int
xfs_parent_init(
struct xfs_mount *mp,
struct xfs_parent_defer **parentp)
{
struct xfs_parent_defer *parent;
int error;
if (!xfs_has_parent(mp))
return 0;
error = xfs_attr_grab_log_assist(mp);
if (error)
return error;
parent = kzalloc(sizeof(*parent), GFP_KERNEL);
if (!parent)
return -ENOMEM;
/* init parent da_args */
parent->args.geo = mp->m_attr_geo;
parent->args.whichfork = XFS_ATTR_FORK;
parent->args.attr_filter = XFS_ATTR_PARENT;
parent->args.op_flags = XFS_DA_OP_OKNOENT | XFS_DA_OP_LOGGED;
parent->args.name = (const uint8_t *)&parent->rec;
parent->args.namelen = sizeof(struct xfs_parent_name_rec);
*parentp = parent;
return 0;
}int
xfs_parent_defer_add(
struct xfs_trans *tp,
struct xfs_parent_defer *parent,
struct xfs_inode *dp,
struct xfs_name *parent_name,
xfs_dir2_dataptr_t parent_offset,
struct xfs_inode *child)
{
struct xfs_da_args *args = &parent->args;
xfs_init_parent_name_rec(&parent->rec, dp, diroffset);
args->hashval = xfs_da_hashname(args->name, args->namelen);
args->trans = tp;
args->dp = child;
if (parent_name) {
args->name = parent_name->name;
args->valuelen = parent_name->len;
}
return xfs_attr_defer_add(args);
}And then the callsites become:
/*
* If we have parent pointers, we now need to add the parent record to
* the attribute fork of the inode. If this is the initial parent
* attribute, we need to create it correctly, otherwise we can just add
* the parent to the inode.
*/
if (parent) {
error = xfs_parent_defer_add(tp, parent, tdp,
target_name, diroffset, sip);
if (error)
goto out_defer_cancel;
}Aside from the API suggestions, the rest looks good to me.
--D
Quoted 29 lines
fs/xfs/xfs_inode.c +11
-2+if (error) + goto out_defer_cancel; + } +/* * If this is a synchronous mount, make sure that the * link transaction goes to disk before returning to @@ -1310,11 +1330,16 @@ xfs_link(xfs_iunlock (sip,XFS_ILOCK_EXCL );return error; - error_return: +out_defer_cancel: +xfs_defer_cancel (tp); +error_return:xfs_trans_cancel (tp);xfs_iunlock (tdp,XFS_ILOCK_EXCL );xfs_iunlock (sip,XFS_ILOCK_EXCL ); - std_return: +drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); +std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;return error;Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 13/18] xfs: add parent attributes to link
Quoted 44 lines · Darrick J. Wong
Quoted 43 lines · Allison Henderson
This patch modifies
xfs_linkto add a parent pointer to the inode.[bfoster: rebase, use VFS inode fields, fix
xfs_bmap_finish()usage]
[achender: rebased, changed__unint32_ttoxfs_dir2_dataptr_t,
fixed null pointer bugs]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_inode.c | 43 ++++++++++++++++++++++++++++++++++------ --- 1 file changed, 34 insertions(+), 9 deletions(-)fs/xfs/xfs_inode.c +6
-4@@ -1228,14 +1228,16 @@ xfs_create_tmpfile(int xfs_link ( -xfs_inode_t *tdp, -xfs_inode_t *sip, +struct xfs_inode *tdp, +struct xfs_inode *sip,struct xfs_name *target_name) { -xfs_mount_t *mp = tdp->i_mount; -xfs_trans_t *tp; +struct xfs_mount *mp = tdp->i_mount; +struct xfs_trans *tp;int error, nospace_error =0 ;int resblks; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_link (tdp, target_name);@@ -1252,11 +1254,17 @@ xfs_link(if (error) goto std_return;
Quoted 6 lines
fs/xfs/xfs_inode.c xfs_link( · line 1258
Quoted 2 lines
fs/xfs/xfs_inode.c +2
-0+if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, sip, target_name, &parent);Why does
xfs_parent_initcheckxfs_has_parentif the callers already do
that?
It was part of the solution outlined in the last review. It is redundant, but not an inappropriate sanity check for that function either. I can remove it from the helper if it bothers folks.
Quoted 9 lines
fs/xfs/xfs_inode.c xfs_link( · line 1263
Quoted 6 lines
fs/xfs/xfs_inode.c +4
-0+if (error) + goto std_return; + } + resblks =XFS_LINK_SPACE_RES (mp, target_name->len);Same comment about increasing
XFS_LINK_SPACE_RESto accomodate xattr expansion as I had for the last patch.
So we do use XFS_LINK_SPACE_RES here, but didnt we update the tr_link below in patch 11 to accommodate for the extra space? Maybe I'm not understanding why we would need both?
Quoted 108 lines
Quoted 36 lines
fs/xfs/xfs_inode.c +1
-1error =xfs_trans_alloc_dir (tdp, &M_RES (mp)->tr_link, sip, &resblks, &tp, &nospace_error);if (error) - goto std_return; + goto drop_incompat;/* fs/xfs/xfs_inode.c +4
-2* If we are using project inheritance, we only allow hard link @@ -1289,14 +1297,26 @@ xfs_link( } error =xfs_dir_createname (tp, tdp, target_name, sip->i_ino, - resblks,NULL ); + resblks, &diroffset);if (error) - goto error_return; + goto out_defer_cancel;xfs_trans_ichgtime (tp, tdp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, tdp,XFS_ILOG_CORE );xfs_bumplink (tp, sip); +/* +* If we have parent pointers, we now need to add the parent record to
fs/xfs/xfs_inode.c +1
-0+* the attribute fork of the inode. If this is the initial parent
fs/xfs/xfs_inode.c +1
-0+* attribute, we need to create it correctly, otherwise we can just add
fs/xfs/xfs_inode.c +4
-0+* the parent to the inode. +*/ +if (parent) { + error = xfs_parent_defer_add(tp, tdp, parent,Quoted 1 line
diroffset);
A followup to the comments I made to the previous patch about
parent->args.dp--Since you're partially initializing the
xfs_defer_parentstructure before you even have the dir offset, why not delay initializing the parent and child pointers until thexfs_parent_defer_addstep?int
xfs_parent_init(
structxfs_mount*mp,
structxfs_parent_defer**parentp){ struct xfs_parent_defer *parent; int error; if (!xfs_has_parent(mp)) return 0; error = xfs_attr_grab_log_assist(mp); if (error) return error; parent = kzalloc(sizeof(*parent), GFP_KERNEL); if (!parent) return -ENOMEM; /* init parent da_args */ parent->args.geo = mp->m_attr_geo; parent->args.whichfork = XFS_ATTR_FORK; parent->args.attr_filter = XFS_ATTR_PARENT; parent->args.op_flags = XFS_DA_OP_OKNOENT | XFS_DA_OP_LOGGED; parent->args.name = (const uint8_t *)&parent->rec; parent->args.namelen = sizeof(struct xfs_parent_name_rec); *parentp = parent; return 0; }int
xfs_parent_defer_add(
structxfs_trans*tp,
structxfs_parent_defer*parent,
structxfs_inode*dp,
structxfs_name*parent_name,
xfs_dir2_dataptr_tparent_offset,
structxfs_inode*child){ struct xfs_da_args *args = &parent->args; xfs_init_parent_name_rec(&parent->rec, dp, diroffset); args->hashval = xfs_da_hashname(args->name, args->namelen); args->trans = tp; args->dp = child; if (parent_name) { args->name = parent_name->name; args->valuelen = parent_name->len; } return xfs_attr_defer_add(args); }And then the callsites become:
/* * If we have parent pointers, we now need to add the parentrecord to
* the attribute fork of the inode. If this is the initial
parent
* attribute, we need to create it correctly, otherwise we can
just add
Quoted 8 lines
* the parent to the inode.
*/
if (parent) {
error = xfs_parent_defer_add(tp, parent, tdp,
target_name, diroffset, sip);
if (error)
goto out_defer_cancel;
}Sure, I can scoot that part down to the defer_add helper. Thanks for the reviews!
Allison
Quoted 34 lines
Aside from the API suggestions, the rest looks good to me.
--D
Quoted 29 lines
fs/xfs/xfs_inode.c +11
-2+if (error) + goto out_defer_cancel; + } +/* * If this is a synchronous mount, make sure that the * link transaction goes to disk before returning to @@ -1310,11 +1330,16 @@ xfs_link(xfs_iunlock (sip,XFS_ILOCK_EXCL );return error; - error_return: +out_defer_cancel: +xfs_defer_cancel (tp); +error_return:xfs_trans_cancel (tp);xfs_iunlock (tdp,XFS_ILOCK_EXCL );xfs_iunlock (sip,XFS_ILOCK_EXCL ); - std_return: +drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); +std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;return error;Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 13/18] xfs: add parent attributes to link
Quoted 65 lines · Alli
Quoted 44 lines · Darrick J. Wong
Quoted 43 lines · Allison Henderson
This patch modifies
xfs_linkto add a parent pointer to the inode.[bfoster: rebase, use VFS inode fields, fix
xfs_bmap_finish()usage]
[achender: rebased, changed__unint32_ttoxfs_dir2_dataptr_t,
fixed null pointer bugs]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_inode.c | 43 ++++++++++++++++++++++++++++++++++------ --- 1 file changed, 34 insertions(+), 9 deletions(-)fs/xfs/xfs_inode.c +6
-4@@ -1228,14 +1228,16 @@ xfs_create_tmpfile(int xfs_link ( -xfs_inode_t *tdp, -xfs_inode_t *sip, +struct xfs_inode *tdp, +struct xfs_inode *sip,struct xfs_name *target_name) { -xfs_mount_t *mp = tdp->i_mount; -xfs_trans_t *tp; +struct xfs_mount *mp = tdp->i_mount; +struct xfs_trans *tp;int error, nospace_error =0 ;int resblks; +xfs_dir2_dataptr_t diroffset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_link (tdp, target_name);@@ -1252,11 +1254,17 @@ xfs_link(if (error) goto std_return;Quoted 6 lines
fs/xfs/xfs_inode.c xfs_link( · line 1258
Quoted 2 lines
fs/xfs/xfs_inode.c +2
-0+if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, sip, target_name, &parent);Why does
xfs_parent_initcheckxfs_has_parentif the callers already do
that?It was part of the solution outlined in the last review. It is redundant, but not an inappropriate sanity check for that function either. I can remove it from the helper if it bothers folks.
Quoted 9 lines
fs/xfs/xfs_inode.c xfs_link( · line 1263
Quoted 6 lines
fs/xfs/xfs_inode.c +4
-0+if (error) + goto std_return; + } + resblks =XFS_LINK_SPACE_RES (mp, target_name->len);Same comment about increasing
XFS_LINK_SPACE_RESto accomodate xattr expansion as I had for the last patch.
Quoted 3 lines
So we do use
XFS_LINK_SPACE_REShere, but didnt we update thetr_linkbelow in patch 11 to accommodate for the extra space? Maybe I'm not understanding why we would need both?
D'oh, I apparently forgot to respond to this. :/
tr_res == space we reserve in the *log* to record updates.
XFS_LINK_SPACE_RES == block we reserve from the filesystem free space to handle expansions of metadata structures.
At this point in this version of the patchset, you've increased the log space reservations in anticipation of logging more information per transaction. However, you've not increased the free space reservations to handle potential node splitting in the ondisk xattr btree.
(Will copy this to my reply for the patch resend.)
--D
Quoted 155 lines
Quoted 108 lines
Quoted 36 lines
fs/xfs/xfs_inode.c +1
-1error =xfs_trans_alloc_dir (tdp, &M_RES (mp)->tr_link, sip, &resblks, &tp, &nospace_error);if (error) - goto std_return; + goto drop_incompat;/* fs/xfs/xfs_inode.c +4
-2* If we are using project inheritance, we only allow hard link @@ -1289,14 +1297,26 @@ xfs_link( } error =xfs_dir_createname (tp, tdp, target_name, sip->i_ino, - resblks,NULL ); + resblks, &diroffset);if (error) - goto error_return; + goto out_defer_cancel;xfs_trans_ichgtime (tp, tdp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, tdp,XFS_ILOG_CORE );xfs_bumplink (tp, sip); +/* +* If we have parent pointers, we now need to add the parent record to
fs/xfs/xfs_inode.c +1
-0+* the attribute fork of the inode. If this is the initial parent
fs/xfs/xfs_inode.c +1
-0+* attribute, we need to create it correctly, otherwise we can just add
fs/xfs/xfs_inode.c +4
-0+* the parent to the inode. +*/ +if (parent) { + error = xfs_parent_defer_add(tp, tdp, parent,Quoted 1 line
diroffset);
A followup to the comments I made to the previous patch about
parent->args.dp--Since you're partially initializing the
xfs_defer_parentstructure before you even have the dir offset, why not delay initializing the parent and child pointers until thexfs_parent_defer_addstep?int
xfs_parent_init(
structxfs_mount*mp,
structxfs_parent_defer**parentp){ struct xfs_parent_defer *parent; int error; if (!xfs_has_parent(mp)) return 0; error = xfs_attr_grab_log_assist(mp); if (error) return error; parent = kzalloc(sizeof(*parent), GFP_KERNEL); if (!parent) return -ENOMEM; /* init parent da_args */ parent->args.geo = mp->m_attr_geo; parent->args.whichfork = XFS_ATTR_FORK; parent->args.attr_filter = XFS_ATTR_PARENT; parent->args.op_flags = XFS_DA_OP_OKNOENT | XFS_DA_OP_LOGGED; parent->args.name = (const uint8_t *)&parent->rec; parent->args.namelen = sizeof(struct xfs_parent_name_rec); *parentp = parent; return 0; }int
xfs_parent_defer_add(
structxfs_trans*tp,
structxfs_parent_defer*parent,
structxfs_inode*dp,
structxfs_name*parent_name,
xfs_dir2_dataptr_tparent_offset,
structxfs_inode*child){ struct xfs_da_args *args = &parent->args; xfs_init_parent_name_rec(&parent->rec, dp, diroffset); args->hashval = xfs_da_hashname(args->name, args->namelen); args->trans = tp; args->dp = child; if (parent_name) { args->name = parent_name->name; args->valuelen = parent_name->len; } return xfs_attr_defer_add(args); }And then the callsites become:
/* * If we have parent pointers, we now need to add the parentrecord to
* the attribute fork of the inode. If this is the initial
parent
* attribute, we need to create it correctly, otherwise we can
just addQuoted 8 lines
* the parent to the inode. */ if (parent) { error = xfs_parent_defer_add(tp, parent, tdp, target_name, diroffset, sip); if (error) goto out_defer_cancel; }Sure, I can scoot that part down to the
defer_addhelper. Thanks for the reviews!Allison
Quoted 34 lines
Aside from the API suggestions, the rest looks good to me.
--D
Quoted 29 lines
fs/xfs/xfs_inode.c +11
-2+if (error) + goto out_defer_cancel; + } +/* * If this is a synchronous mount, make sure that the * link transaction goes to disk before returning to @@ -1310,11 +1330,16 @@ xfs_link(xfs_iunlock (sip,XFS_ILOCK_EXCL );return error; - error_return: +out_defer_cancel: +xfs_defer_cancel (tp); +error_return:xfs_trans_cancel (tp);xfs_iunlock (tdp,XFS_ILOCK_EXCL );xfs_iunlock (sip,XFS_ILOCK_EXCL ); - std_return: +drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); +std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;return error;Signature · 1 line
2.25.1
PATCH RESEND v2 14/18 xfs: remove parent pointers in unlink
In reply to Allison HendersonThis patch removes the parent pointer attribute during unlink
[bfoster: rebase, use VFS inode generation] [achender: rebased, changed __unint32_t to xfs_dir2_dataptr_t
implemented xfs_attr_remove_parent]
--- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr.h | 1 + fs/xfs/libxfs/xfs_parent.c | 15 +++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 3 +++ fs/xfs/xfs_inode.c | 29 +++++++++++++++++++++++------ 5 files changed, 43 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
@@ -936,7 +936,7 @@ xfs_attr_defer_replace( }/* Removes an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_remove (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +1 -0
@@ -545,6 +545,7 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args); +int xfs_attr_defer_remove (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
fs/xfs/libxfs/xfs_parent.c +15 -0
@@ -123,6 +123,21 @@ xfs_parent_defer_add(return xfs_attr_defer_add (args); } +int +xfs_parent_defer_remove ( +struct xfs_trans *tp, +struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, ip, diroffset); + args->trans = tp; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_remove (args); +} + void xfs_parent_cancel( xfs_mount_t *mp,
fs/xfs/libxfs/xfs_parent.h +3 -0
@@ -29,6 +29,9 @@ int xfs_parent_init(xfs_mount_t *mp, xfs_inode_t *ip,int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_inode *ip,struct xfs_parent_defer *parent,xfs_dir2_dataptr_t diroffset); +int xfs_parent_defer_remove (struct xfs_trans *tp,struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset);void xfs_parent_cancel (xfs_mount_t *mp,struct xfs_parent_defer *parent);#endif /* __XFS_PARENT_H__ */
fs/xfs/xfs_inode.c +23 -6
@@ -2464,16 +2464,18 @@ xfs_iunpin_wait(*/ int xfs_remove ( -xfs_inode_t *dp, +struct xfs_inode *dp,struct xfs_name *name, -xfs_inode_t *ip) +struct xfs_inode *ip) { -xfs_mount_t *mp = dp->i_mount; -xfs_trans_t *tp =NULL ; +struct xfs_mount *mp = dp->i_mount; +struct xfs_trans *tp =NULL ;int is_dir =S_ISDIR (VFS_I (ip)->i_mode);int dontcare;int error =0 ;uint resblks; +xfs_dir2_dataptr_t dir_offset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_remove (dp, name);
@@ -2488,6 +2490,12 @@ xfs_remove(if (error) goto std_return; +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, ip,NULL , &parent); +if (error) + goto std_return; + } +/* * We try to get the real space reservation first, allowing for * directory btree deletion(s) implying possible bmap insert(s). If we
@@ -2504,7 +2512,7 @@ xfs_remove( &tp, &dontcare);if (error) {ASSERT (error != -ENOSPC ); - goto std_return; + goto drop_incompat; }/*
@@ -2558,12 +2566,18 @@ xfs_remove(if (error) goto out_trans_cancel; - error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks,NULL ); + error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks, &dir_offset);if (error) {ASSERT (error != -ENOENT ); goto out_trans_cancel; } +if (xfs_has_parent (mp)) { + error =xfs_parent_defer_remove (tp, dp, parent, dir_offset); +if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * remove transaction goes to disk before returning to
@@ -2588,6 +2602,9 @@ xfs_remove( out_unlock:xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); std_return:return error; }
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 14/18] xfs: remove parent pointers in unlink
Quoted 46 lines · Allison Henderson
This patch removes the parent pointer attribute during unlink
[bfoster: rebase, use VFS inode generation] [achender: rebased, changed
__unint32_ttoxfs_dir2_dataptr_t
implementedxfs_attr_remove_parent]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr.h | 1 + fs/xfs/libxfs/xfs_parent.c | 15 +++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 3 +++ fs/xfs/xfs_inode.c | 29 +++++++++++++++++++++++------ 5 files changed, 43 insertions(+), 7 deletions(-)fs/xfs/libxfs/xfs_attr.c +1
-1@@ -936,7 +936,7 @@ xfs_attr_defer_replace( }/* Removes an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_remove (struct xfs_da_args *args) {fs/xfs/libxfs/xfs_attr.h +1
-0@@ -545,6 +545,7 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args); +int xfs_attr_defer_remove (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);fs/xfs/libxfs/xfs_parent.c +0
-0
fs/xfs/libxfs/xfs_parent.c xfs_parent_defer_add( · line 131
Quoted 9 lines
fs/xfs/libxfs/xfs_parent.c +6
-0@@ -123,6 +123,21 @@ xfs_parent_defer_add(return xfs_attr_defer_add (args); } +int +xfs_parent_defer_remove ( +struct xfs_trans *tp, +struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset)
Same suggestion about setting args->dp here instead of in xfs_parent_init.
Quoted 78 lines
fs/xfs/libxfs/xfs_parent.c +9
-0+{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, ip, diroffset); + args->trans = tp; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_remove (args); +} + void xfs_parent_cancel( xfs_mount_t *mp,fs/xfs/libxfs/xfs_parent.h +3
-0@@ -29,6 +29,9 @@ int xfs_parent_init(xfs_mount_t *mp, xfs_inode_t *ip,int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_inode *ip,struct xfs_parent_defer *parent,xfs_dir2_dataptr_t diroffset); +int xfs_parent_defer_remove (struct xfs_trans *tp,struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset);void xfs_parent_cancel (xfs_mount_t *mp,struct xfs_parent_defer *parent);#endif /* __XFS_PARENT_H__ */ fs/xfs/xfs_inode.c +13
-5@@ -2464,16 +2464,18 @@ xfs_iunpin_wait(*/ int xfs_remove ( -xfs_inode_t *dp, +struct xfs_inode *dp,struct xfs_name *name, -xfs_inode_t *ip) +struct xfs_inode *ip) { -xfs_mount_t *mp = dp->i_mount; -xfs_trans_t *tp =NULL ; +struct xfs_mount *mp = dp->i_mount; +struct xfs_trans *tp =NULL ;int is_dir =S_ISDIR (VFS_I (ip)->i_mode);int dontcare;int error =0 ;uint resblks; +xfs_dir2_dataptr_t dir_offset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_remove (dp, name);@@ -2488,6 +2490,12 @@ xfs_remove(if (error) goto std_return; +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, ip,NULL , &parent); +if (error) + goto std_return; + } +/* * We try to get the real space reservation first, allowing for * directory btree deletion(s) implying possible bmap insert(s). If we @@ -2504,7 +2512,7 @@ xfs_remove( &tp, &dontcare);if (error) {ASSERT (error != -ENOSPC ); - goto std_return; + goto drop_incompat; }/* @@ -2558,12 +2566,18 @@ xfs_remove(if (error) goto out_trans_cancel;
fs/xfs/xfs_inode.c xfs_remove( · line 2576
Quoted 10 lines
fs/xfs/xfs_inode.c +3
-1- error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks,NULL ); + error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks, &dir_offset);if (error) {ASSERT (error != -ENOENT ); goto out_trans_cancel; } +if (xfs_has_parent (mp)) { + error =xfs_parent_defer_remove (tp, dp, parent, dir_offset);
If it's safe to gate xfs_parent_cancel on "if (parent)" then can we avoid the atomic bit access by doing that here too?
Otherwise looks good here.
--D
Quoted 20 lines
fs/xfs/xfs_inode.c +7
-0+if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * remove transaction goes to disk before returning to @@ -2588,6 +2602,9 @@ xfs_remove( out_unlock:xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); std_return:return error; }Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 14/18] xfs: remove parent pointers in unlink
Quoted 51 lines · Darrick J. Wong
Quoted 50 lines · Allison Henderson
This patch removes the parent pointer attribute during unlink
[bfoster: rebase, use VFS inode generation] [achender: rebased, changed
__unint32_ttoxfs_dir2_dataptr_t
implementedxfs_attr_remove_parent]Signed-off-by Dave Chinner <[email protected]>Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr.h | 1 + fs/xfs/libxfs/xfs_parent.c | 15 +++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 3 +++ fs/xfs/xfs_inode.c | 29 +++++++++++++++++++++++------ 5 files changed, 43 insertions(+), 7 deletions(-)fs/xfs/libxfs/xfs_attr.c +1
-1@@ -936,7 +936,7 @@ xfs_attr_defer_replace( }/* Removes an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_remove (struct xfs_da_args *args) {fs/xfs/libxfs/xfs_attr.h +1
-0@@ -545,6 +545,7 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args); +int xfs_attr_defer_remove (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);fs/xfs/libxfs/xfs_parent.c +0
-0@@ -123,6 +123,21 @@ xfs_parent_defer_add(return xfs_attr_defer_add (args); }
Quoted 9 lines
fs/xfs/libxfs/xfs_parent.c xfs_parent_defer_add( · line 131
Quoted 6 lines
fs/xfs/libxfs/xfs_parent.c +6
-0+int +xfs_parent_defer_remove ( +struct xfs_trans *tp, +struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset)Same suggestion about setting
args->dphere instead of inxfs_parent_init.
Sure, that should be fine.
Quoted 95 lines
Quoted 95 lines
fs/xfs/libxfs/xfs_parent.c +9
-0+{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, ip, diroffset); + args->trans = tp; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_remove (args); +} + void xfs_parent_cancel( xfs_mount_t *mp,fs/xfs/libxfs/xfs_parent.h +3
-0@@ -29,6 +29,9 @@ int xfs_parent_init(xfs_mount_t *mp, xfs_inode_t *ip,int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_inode *ip,struct xfs_parent_defer *parent,xfs_dir2_dataptr_t diroffset); +int xfs_parent_defer_remove (struct xfs_trans *tp,struct xfs_inode *ip, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset);void xfs_parent_cancel(xfs_mount_t *mp, struct
xfs_parent_defer
*parent);#endif /*
__XFS_PARENT_H__*/fs/xfs/xfs_inode.c +12
-4@@ -2464,16 +2464,18 @@ xfs_iunpin_wait(*/ int xfs_remove ( -xfs_inode_t *dp, +struct xfs_inode *dp,struct xfs_name *name, -xfs_inode_t *ip) +struct xfs_inode *ip) { -xfs_mount_t *mp = dp->i_mount; -xfs_trans_t *tp =NULL ; +struct xfs_mount *mp = dp->i_mount; +struct xfs_trans *tp =NULL ;int is_dir =S_ISDIR (VFS_I (ip)->i_mode);int dontcare;int error =0 ;uint resblks; +xfs_dir2_dataptr_t dir_offset; +struct xfs_parent_defer *parent =NULL ;trace_xfs_remove (dp, name);@@ -2488,6 +2490,12 @@ xfs_remove(if (error) goto std_return; +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, ip,NULL , &parent); +if (error) + goto std_return; + } +/* * We try to get the real space reservation first, allowing for * directory btree deletion(s) implying possible bmap insert(s). If we
fs/xfs/xfs_inode.c +2
-2@@ -2504,7 +2512,7 @@ xfs_remove( &tp, &dontcare);if (error) {ASSERT (error != -ENOSPC ); - goto std_return; + goto drop_incompat; }/* @@ -2558,12 +2566,18 @@ xfs_remove(if (error) goto out_trans_cancel; - error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks, NULL); + error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks, &dir_offset);if (error) {ASSERT (error != -ENOENT ); goto out_trans_cancel; }
Quoted 6 lines
fs/xfs/xfs_inode.c xfs_remove( · line 2579
Quoted 3 lines
fs/xfs/xfs_inode.c +2
-0+if (xfs_has_parent (mp)) { + error =xfs_parent_defer_remove (tp, dp, parent, dir_offset);If it's safe to gate
xfs_parent_cancelon "if (parent)" then can we avoid the atomic bit access by doing that here too?
Oh, sure, I likely just forgot to update that conditional.
Quoted 2 lines
Otherwise looks good here.
Thank you!
Allison
Quoted 23 lines
--D
Quoted 20 lines
fs/xfs/xfs_inode.c +4
-0+if (error) + goto out_trans_cancel; + } +fs/xfs/xfs_inode.c +3
-0/* * If this is a synchronous mount, make sure that the * remove transaction goes to disk before returning to @@ -2588,6 +2602,9 @@ xfs_remove( out_unlock:xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); + drop_incompat: +if (parent) +xfs_parent_cancel (mp, parent); std_return:return error; }Signature · 1 line
2.25.1
PATCH RESEND v2 15/18 xfs: Add parent pointers to rename
In reply to Allison HendersonThis patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer. In the case of xfs_cross_rename, we modify the routine not to roll the transaction just yet. We will do this after the parent pointer is added in the calling xfs_rename function.
--- fs/xfs/xfs_inode.c | 128 +++++++++++++++++++++++++++++++++------------ 1 file changed, 94 insertions(+), 34 deletions(-)
fs/xfs/xfs_inode.c +94 -34
@@ -2776,7 +2776,7 @@ xfs_cross_rename( }xfs_trans_ichgtime (tp, dp1,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, dp1,XFS_ILOG_CORE ); -return xfs_finish_rename (tp); +return 0 ; out_trans_abort:xfs_trans_cancel (tp);
@@ -2834,26 +2834,31 @@ xfs_rename_alloc_whiteout(*/ int xfs_rename ( -struct user_namespace *mnt_userns, -struct xfs_inode *src_dp, -struct xfs_name *src_name, -struct xfs_inode *src_ip, -struct xfs_inode *target_dp, -struct xfs_name *target_name, -struct xfs_inode *target_ip, -unsigned int flags) +struct user_namespace *mnt_userns, +struct xfs_inode *src_dp, +struct xfs_name *src_name, +struct xfs_inode *src_ip, +struct xfs_inode *target_dp, +struct xfs_name *target_name, +struct xfs_inode *target_ip, +unsigned int flags) { -struct xfs_mount *mp = src_dp->i_mount; -struct xfs_trans *tp; -struct xfs_inode *wip =NULL ;/* whiteout inode */ -struct xfs_inode *inodes[__XFS_SORT_INODES]; -int i; -int num_inodes = __XFS_SORT_INODES; -bool new_parent = (src_dp != target_dp); -bool src_is_directory =S_ISDIR (VFS_I (src_ip)->i_mode); -int spaceres; -bool retried = false; -int error, nospace_error =0 ; +struct xfs_mount *mp = src_dp->i_mount; +struct xfs_trans *tp; +struct xfs_inode *wip =NULL ;/* whiteout inode */ +struct xfs_inode *inodes[__XFS_SORT_INODES]; +int i; +int num_inodes = __XFS_SORT_INODES; +bool new_parent = (src_dp != target_dp); +bool src_is_directory =S_ISDIR (VFS_I (src_ip)->i_mode); +int spaceres; +bool retried = false; +int error, nospace_error =0 ; +xfs_dir2_dataptr_t new_diroffset; +xfs_dir2_dataptr_t old_diroffset; +struct xfs_parent_defer *old_parent_ptr =NULL ; +struct xfs_parent_defer *new_parent_ptr =NULL ; +struct xfs_parent_defer *target_parent_ptr =NULL ;trace_xfs_rename (src_dp, target_dp, src_name, target_name);
@@ -2877,6 +2882,15 @@ xfs_rename(xfs_sort_for_rename (src_dp, target_dp, src_ip, target_ip, wip, inodes, &num_inodes); +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, src_ip,NULL , &old_parent_ptr); +if (error) + goto out_release_wip; + error =xfs_parent_init (mp, src_ip, target_name, + &new_parent_ptr); +if (error) + goto out_release_wip; + } retry: nospace_error =0 ;
@@ -2889,7 +2903,7 @@ xfs_rename( &tp); }if (error) - goto out_release_wip; + goto drop_incompat;/* * Attach the dquots to the inodes
@@ -2911,14 +2925,14 @@ xfs_rename(* we can rely on either trans_commit or trans_cancel to unlock * them. */ -xfs_trans_ijoin (tp, src_dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, src_dp,0 );if (new_parent) -xfs_trans_ijoin (tp, target_dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, src_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_dp,0 ); +xfs_trans_ijoin (tp, src_ip,0 );if (target_ip) -xfs_trans_ijoin (tp, target_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_ip,0 );if (wip) -xfs_trans_ijoin (tp, wip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, wip,0 );/* * If we are using project inheritance, we only allow renames
@@ -2928,15 +2942,16 @@ xfs_rename(if (unlikely ((target_dp->i_diflags &XFS_DIFLAG_PROJINHERIT ) && target_dp->i_projid != src_ip->i_projid)) { error = -EXDEV ; - goto out_trans_cancel; + goto out_unlock; }/* RENAME_EXCHANGE is unique from here on. */ -if (flags &RENAME_EXCHANGE ) -return xfs_cross_rename (tp, src_dp, src_name, src_ip, +if (flags &RENAME_EXCHANGE ) { + error =xfs_cross_rename (tp, src_dp, src_name, src_ip, target_dp, target_name, target_ip, spaceres); - + goto out_pptr; + }/* * Try to reserve quota to handle an expansion of the target directory. * We'll allow the rename to continue in reservationless mode if we hit
@@ -3052,7 +3067,7 @@ xfs_rename(* to account for the ".." reference from the new entry. */ error =xfs_dir_createname (tp, target_dp, target_name, - src_ip->i_ino, spaceres,NULL ); + src_ip->i_ino, spaceres, &new_diroffset);if (error) goto out_trans_cancel;
@@ -3073,10 +3088,14 @@ xfs_rename(* name at the destination directory, remove it first. */ error =xfs_dir_replace (tp, target_dp, target_name, - src_ip->i_ino, spaceres,NULL ); + src_ip->i_ino, spaceres, &new_diroffset);if (error) goto out_trans_cancel; +if (xfs_has_parent (mp)) + error =xfs_parent_init (mp, target_ip,NULL , + &target_parent_ptr); +xfs_trans_ichgtime (tp, target_dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );
@@ -3146,26 +3165,67 @@ xfs_rename(*/ if (wip) error =xfs_dir_replace (tp, src_dp, src_name, wip->i_ino, - spaceres,NULL ); + spaceres, &old_diroffset);else error =xfs_dir_removename (tp, src_dp, src_name, src_ip->i_ino, - spaceres,NULL ); + spaceres, &old_diroffset);if (error) goto out_trans_cancel; +out_pptr: +if (new_parent_ptr) { + error =xfs_parent_defer_add (tp, target_dp, new_parent_ptr, + new_diroffset); +if (error) + goto out_trans_cancel; + } + +if (old_parent_ptr) { + error =xfs_parent_defer_remove (tp, src_dp, old_parent_ptr, + old_diroffset); +if (error) + goto out_trans_cancel; + } + +if (target_parent_ptr) { + error =xfs_parent_defer_remove (tp, target_dp, + target_parent_ptr, + new_diroffset); +if (error) + goto out_trans_cancel; + } +xfs_trans_ichgtime (tp, src_dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, src_dp,XFS_ILOG_CORE );if (new_parent)xfs_trans_log_inode (tp, target_dp,XFS_ILOG_CORE ); error =xfs_finish_rename (tp); + +out_unlock:if (wip)xfs_irele (wip); +if (wip) +xfs_iunlock (wip,XFS_ILOCK_EXCL ); +if (target_ip) +xfs_iunlock (target_ip,XFS_ILOCK_EXCL ); +xfs_iunlock (src_ip,XFS_ILOCK_EXCL ); +if (new_parent) +xfs_iunlock (target_dp,XFS_ILOCK_EXCL ); +xfs_iunlock (src_dp,XFS_ILOCK_EXCL ); +return error; out_trans_cancel:xfs_trans_cancel (tp); +drop_incompat: +if (new_parent_ptr) +xfs_parent_cancel (mp, new_parent_ptr); +if (old_parent_ptr) +xfs_parent_cancel (mp, old_parent_ptr); +if (target_parent_ptr) +xfs_parent_cancel (mp, target_parent_ptr); out_release_wip:if (wip)xfs_irele (wip);
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 15/18] xfs: Add parent pointers to rename
Quoted 211 lines · Allison Henderson
This patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer. In the case of
xfs_cross_rename, we modify the routine not to roll the transaction just yet. We will do this after the parent pointer is added in the callingxfs_renamefunction.Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_inode.c | 128 +++++++++++++++++++++++++++++++++------------ 1 file changed, 94 insertions(+), 34 deletions(-)fs/xfs/xfs_inode.c +78
-34@@ -2776,7 +2776,7 @@ xfs_cross_rename( }xfs_trans_ichgtime (tp, dp1,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, dp1,XFS_ILOG_CORE ); -return xfs_finish_rename (tp); +return 0 ; out_trans_abort:xfs_trans_cancel (tp);@@ -2834,26 +2834,31 @@ xfs_rename_alloc_whiteout(*/ int xfs_rename ( -struct user_namespace *mnt_userns, -struct xfs_inode *src_dp, -struct xfs_name *src_name, -struct xfs_inode *src_ip, -struct xfs_inode *target_dp, -struct xfs_name *target_name, -struct xfs_inode *target_ip, -unsigned int flags) +struct user_namespace *mnt_userns, +struct xfs_inode *src_dp, +struct xfs_name *src_name, +struct xfs_inode *src_ip, +struct xfs_inode *target_dp, +struct xfs_name *target_name, +struct xfs_inode *target_ip, +unsigned int flags) { -struct xfs_mount *mp = src_dp->i_mount; -struct xfs_trans *tp; -struct xfs_inode *wip =NULL ;/* whiteout inode */ -struct xfs_inode *inodes[__XFS_SORT_INODES]; -int i; -int num_inodes = __XFS_SORT_INODES; -bool new_parent = (src_dp != target_dp); -bool src_is_directory =S_ISDIR (VFS_I (src_ip)->i_mode); -int spaceres; -bool retried = false; -int error, nospace_error =0 ; +struct xfs_mount *mp = src_dp->i_mount; +struct xfs_trans *tp; +struct xfs_inode *wip =NULL ;/* whiteout inode */ +struct xfs_inode *inodes[__XFS_SORT_INODES]; +int i; +int num_inodes = __XFS_SORT_INODES; +bool new_parent = (src_dp != target_dp); +bool src_is_directory =S_ISDIR (VFS_I (src_ip)->i_mode); +int spaceres; +bool retried = false; +int error, nospace_error =0 ; +xfs_dir2_dataptr_t new_diroffset; +xfs_dir2_dataptr_t old_diroffset; +struct xfs_parent_defer *old_parent_ptr =NULL ; +struct xfs_parent_defer *new_parent_ptr =NULL ; +struct xfs_parent_defer *target_parent_ptr =NULL ;trace_xfs_rename (src_dp, target_dp, src_name, target_name);@@ -2877,6 +2882,15 @@ xfs_rename(xfs_sort_for_rename (src_dp, target_dp, src_ip, target_ip, wip, inodes, &num_inodes); +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, src_ip,NULL , &old_parent_ptr); +if (error) + goto out_release_wip; + error =xfs_parent_init (mp, src_ip, target_name, + &new_parent_ptr); +if (error) + goto out_release_wip; + } retry: nospace_error =0 ;@@ -2889,7 +2903,7 @@ xfs_rename( &tp); }if (error) - goto out_release_wip; + goto drop_incompat;/* * Attach the dquots to the inodes @@ -2911,14 +2925,14 @@ xfs_rename(* we can rely on either trans_commit or trans_cancel to unlock * them. */ -xfs_trans_ijoin (tp, src_dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, src_dp,0 );if (new_parent) -xfs_trans_ijoin (tp, target_dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, src_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_dp,0 ); +xfs_trans_ijoin (tp, src_ip,0 );if (target_ip) -xfs_trans_ijoin (tp, target_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_ip,0 );if (wip) -xfs_trans_ijoin (tp, wip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, wip,0 );/* * If we are using project inheritance, we only allow renames @@ -2928,15 +2942,16 @@ xfs_rename(if (unlikely ((target_dp->i_diflags &XFS_DIFLAG_PROJINHERIT ) && target_dp->i_projid != src_ip->i_projid)) { error = -EXDEV ; - goto out_trans_cancel; + goto out_unlock; }/* RENAME_EXCHANGE is unique from here on. */ -if (flags &RENAME_EXCHANGE ) -return xfs_cross_rename (tp, src_dp, src_name, src_ip, +if (flags &RENAME_EXCHANGE ) { + error =xfs_cross_rename (tp, src_dp, src_name, src_ip, target_dp, target_name, target_ip, spaceres); - + goto out_pptr; + }/* * Try to reserve quota to handle an expansion of the target directory. * We'll allow the rename to continue in reservationless mode if we hit @@ -3052,7 +3067,7 @@ xfs_rename(* to account for the ".." reference from the new entry. */ error =xfs_dir_createname (tp, target_dp, target_name, - src_ip->i_ino, spaceres,NULL ); + src_ip->i_ino, spaceres, &new_diroffset);if (error) goto out_trans_cancel;@@ -3073,10 +3088,14 @@ xfs_rename(* name at the destination directory, remove it first. */ error =xfs_dir_replace (tp, target_dp, target_name, - src_ip->i_ino, spaceres,NULL ); + src_ip->i_ino, spaceres, &new_diroffset);if (error) goto out_trans_cancel; +if (xfs_has_parent (mp)) + error =xfs_parent_init (mp, target_ip,NULL , + &target_parent_ptr); +xfs_trans_ichgtime (tp, target_dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );@@ -3146,26 +3165,67 @@ xfs_rename(*/ if (wip) error =xfs_dir_replace (tp, src_dp, src_name, wip->i_ino, - spaceres,NULL ); + spaceres, &old_diroffset);else error =xfs_dir_removename (tp, src_dp, src_name, src_ip->i_ino, - spaceres,NULL ); + spaceres, &old_diroffset);if (error) goto out_trans_cancel; +out_pptr: +if (new_parent_ptr) { + error =xfs_parent_defer_add (tp, target_dp, new_parent_ptr, + new_diroffset); +if (error) + goto out_trans_cancel; + } + +if (old_parent_ptr) { + error =xfs_parent_defer_remove (tp, src_dp, old_parent_ptr, + old_diroffset); +if (error) + goto out_trans_cancel; + } + +if (target_parent_ptr) { + error =xfs_parent_defer_remove (tp, target_dp, + target_parent_ptr, + new_diroffset); +if (error) + goto out_trans_cancel; + } +xfs_trans_ichgtime (tp, src_dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, src_dp,XFS_ILOG_CORE );if (new_parent)xfs_trans_log_inode (tp, target_dp,XFS_ILOG_CORE ); error =xfs_finish_rename (tp); + +out_unlock:
fs/xfs/xfs_inode.c xfs_rename( · line 3216
Quoted 10 lines
fs/xfs/xfs_inode.c +8
-0if (wip)xfs_irele (wip); +if (wip) +xfs_iunlock (wip,XFS_ILOCK_EXCL ); +if (target_ip) +xfs_iunlock (target_ip,XFS_ILOCK_EXCL ); +xfs_iunlock (src_ip,XFS_ILOCK_EXCL ); +if (new_parent) +xfs_iunlock (target_dp,XFS_ILOCK_EXCL ); +xfs_iunlock (src_dp,XFS_ILOCK_EXCL );
Sorry to be fussy, but could you separate the ILOCK unlocking changes (and maybe the variable indentation part too) into a separate prep patch, please?
Also, who frees the xfs_parent_defer objects?
--D
Quoted 18 lines
fs/xfs/xfs_inode.c +8
-0+return error; out_trans_cancel:xfs_trans_cancel (tp); +drop_incompat: +if (new_parent_ptr) +xfs_parent_cancel (mp, new_parent_ptr); +if (old_parent_ptr) +xfs_parent_cancel (mp, old_parent_ptr); +if (target_parent_ptr) +xfs_parent_cancel (mp, target_parent_ptr); out_release_wip:if (wip)xfs_irele (wip);Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 15/18] xfs: Add parent pointers to rename
Quoted 242 lines · Darrick J. Wong
Quoted 241 lines · Allison Henderson
This patch removes the old parent pointer attribute during the rename
operation, and re-adds the updated parent pointer. In the case ofxfs_cross_rename, we modify the routine not to roll the transaction just
yet. We will do this after the parent pointer is added in the callingxfs_renamefunction.Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/xfs_inode.c | 128 +++++++++++++++++++++++++++++++++------ ------ 1 file changed, 94 insertions(+), 34 deletions(-)fs/xfs/xfs_inode.c +1
-1@@ -2776,7 +2776,7 @@ xfs_cross_rename( }xfs_trans_ichgtime (tp, dp1,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, dp1,XFS_ILOG_CORE ); -return xfs_finish_rename (tp); +return 0 ; out_trans_abort:fs/xfs/xfs_inode.c +11
-19xfs_trans_cancel (tp);@@ -2834,26 +2834,31 @@ xfs_rename_alloc_whiteout(*/ int xfs_rename ( -struct user_namespace *mnt_userns, -struct xfs_inode *src_dp, -struct xfs_name *src_name, -struct xfs_inode *src_ip, -struct xfs_inode *target_dp, -struct xfs_name *target_name, -struct xfs_inode *target_ip, -unsigned int flags) +struct user_namespace *mnt_userns, +struct xfs_inode *src_dp, +struct xfs_name *src_name, +struct xfs_inode *src_ip, +struct xfs_inode *target_dp, +struct xfs_name *target_name, +struct xfs_inode *target_ip, +unsigned int flags) { -struct xfs_mount *mp = src_dp->i_mount; -struct xfs_trans *tp; -struct xfs_inode *wip =NULL ;/* whiteout inode */ -struct xfs_inode *inodes[__XFS_SORT_INODES]; -int i; -int num_inodes = __XFS_SORT_INODES; -bool new_parent = (src_dp != target_dp); -bool src_is_directory =S_ISDIR (VFS_I (src_ip)->i_mode); -int spaceres; -bool retried = false; -int error, nospace_error =0 ; +struct xfs_mount *mp = src_dp->i_mount; +struct xfs_trans *tp; +struct xfs_inode *wip =NULL ;/* whiteout inode */
fs/xfs/xfs_inode.c +4
-0+struct xfs_inode *inodes[__XFS_SORT_INODES]; +int i; +int num_inodes = __XFS_SORT_INODES; + bool new_parent = (src_dp !=
target_dp);fs/xfs/xfs_inode.c +1
-0+ bool src_is_directory =S_ISDIR(VFS_I(src_ip)->i_mode);
fs/xfs/xfs_inode.c +17
-0+int spaceres; +bool retried = false; +int error, nospace_error =0 ; +xfs_dir2_dataptr_t new_diroffset; +xfs_dir2_dataptr_t old_diroffset; +struct xfs_parent_defer *old_parent_ptr =NULL ; +struct xfs_parent_defer *new_parent_ptr =NULL ; +struct xfs_parent_defer *target_parent_ptr =NULL ;trace_xfs_rename (src_dp, target_dp, src_name, target_name);@@ -2877,6 +2882,15 @@ xfs_rename(xfs_sort_for_rename (src_dp, target_dp, src_ip, target_ip, wip, inodes, &num_inodes); +if (xfs_has_parent (mp)) { + error =xfs_parent_init (mp, src_ip,NULL , &old_parent_ptr); +if (error) + goto out_release_wip; + error =xfs_parent_init (mp, src_ip, target_name, + &new_parent_ptr); +if (error) + goto out_release_wip; + } retry:fs/xfs/xfs_inode.c +11
-10nospace_error =0 ;@@ -2889,7 +2903,7 @@ xfs_rename( &tp); }if (error) - goto out_release_wip; + goto drop_incompat;/* * Attach the dquots to the inodes @@ -2911,14 +2925,14 @@ xfs_rename(* we can rely on either trans_commit or trans_cancel to unlock * them. */ -xfs_trans_ijoin (tp, src_dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, src_dp,0 );if (new_parent) -xfs_trans_ijoin (tp, target_dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, src_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_dp,0 ); +xfs_trans_ijoin (tp, src_ip,0 );if (target_ip) -xfs_trans_ijoin (tp, target_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_ip,0 );if (wip) -xfs_trans_ijoin (tp, wip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, wip,0 );/* * If we are using project inheritance, we only allow renames @@ -2928,15 +2942,16 @@ xfs_rename(if (unlikely ((target_dp->i_diflags &XFS_DIFLAG_PROJINHERIT ) && target_dp->i_projid != src_ip->i_projid)) { error = -EXDEV ; - goto out_trans_cancel; + goto out_unlock; }/* RENAME_EXCHANGE is unique from here on. */ -if (flags &RENAME_EXCHANGE ) -return xfs_cross_rename (tp, src_dp, src_name, src_ip, +if (flags &RENAME_EXCHANGE ) { + error =xfs_cross_rename (tp, src_dp, src_name, src_ip, target_dp, target_name, target_ip, spaceres); - + goto out_pptr; + }/* * Try to reserve quota to handle an expansion of the target directory.
* We'll allow the rename to continue in reservationless mode
if we hitfs/xfs/xfs_inode.c +0
-0@@ -3052,7 +3067,7 @@ xfs_rename(* to account for the ".." reference from the new entry.
fs/xfs/xfs_inode.c +0
-1*/ error = xfs_dir_createname(tp, target_dp, target_name, - src_ip->i_ino, spaceres,NULL);
fs/xfs/xfs_inode.c +1
-0+ src_ip->i_ino, spaceres,
&new_diroffset);fs/xfs/xfs_inode.c +5
-1if (error) goto out_trans_cancel;@@ -3073,10 +3088,14 @@ xfs_rename(* name at the destination directory, remove it first. */ error =xfs_dir_replace (tp, target_dp, target_name, - src_ip->i_ino, spaceres,NULL ); + src_ip->i_ino, spaceres, &new_diroffset);if (error) goto out_trans_cancel; +if (xfs_has_parent (mp)) + error =xfs_parent_init (mp, target_ip,NULL , + &target_parent_ptr); + xfs_trans_ichgtime(tp, target_dp, XFS_ICHGTIME_MOD |
XFS_ICHGTIME_CHG);fs/xfs/xfs_inode.c +31
-2@@ -3146,26 +3165,67 @@ xfs_rename(*/ if (wip) error =xfs_dir_replace (tp, src_dp, src_name, wip- >i_ino, - spaceres,NULL ); + spaceres, &old_diroffset);else error =xfs_dir_removename (tp, src_dp, src_name, src_ip->i_ino, - spaceres,NULL ); + spaceres, &old_diroffset);if (error) goto out_trans_cancel; +out_pptr: +if (new_parent_ptr) { + error =xfs_parent_defer_add (tp, target_dp, new_parent_ptr, + new_diroffset); +if (error) + goto out_trans_cancel; + } + +if (old_parent_ptr) { + error =xfs_parent_defer_remove (tp, src_dp, old_parent_ptr, + old_diroffset); +if (error) + goto out_trans_cancel; + } + +if (target_parent_ptr) { + error =xfs_parent_defer_remove (tp, target_dp, + target_parent_ptr, + new_diroffset); +if (error) + goto out_trans_cancel; + } +xfs_trans_ichgtime (tp, src_dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, src_dp,XFS_ILOG_CORE );if (new_parent)xfs_trans_log_inode (tp, target_dp,XFS_ILOG_CORE ); error =xfs_finish_rename (tp); + +out_unlock:if (wip)xfs_irele (wip); +if (wip) +xfs_iunlock (wip,XFS_ILOCK_EXCL ); +if (target_ip) +xfs_iunlock (target_ip,XFS_ILOCK_EXCL );
Quoted 8 lines
fs/xfs/xfs_inode.c xfs_rename( · line 3216
Quoted 4 lines
fs/xfs/xfs_inode.c +4
-0+xfs_iunlock (src_ip,XFS_ILOCK_EXCL ); +if (new_parent) +xfs_iunlock (target_dp,XFS_ILOCK_EXCL ); +xfs_iunlock (src_dp,XFS_ILOCK_EXCL );Sorry to be fussy, but could you separate the ILOCK unlocking changes (and maybe the variable indentation part too) into a separate prep patch, please?
Sure, that should be fine.
Quoted 3 lines
Also, who frees the
xfs_parent_deferobjects?
its the xfs_parent_cancel() calls below
Quoted 20 lines
--D
Quoted 18 lines
fs/xfs/xfs_inode.c +8
-0+return error; out_trans_cancel:xfs_trans_cancel (tp); +drop_incompat: +if (new_parent_ptr) +xfs_parent_cancel (mp, new_parent_ptr); +if (old_parent_ptr) +xfs_parent_cancel (mp, old_parent_ptr); +if (target_parent_ptr) +xfs_parent_cancel (mp, target_parent_ptr); out_release_wip:if (wip)xfs_irele (wip);Signature · 1 line
2.25.1
PATCH RESEND v2 16/18 xfs: Add the parent pointer support to the superblock version 5.
In reply to Allison Henderson[dchinner: forward ported and cleaned up] [achender: rebased and added parent pointer attribute to
compatible attributes mask]
--- fs/xfs/libxfs/xfs_format.h | 4 +++- fs/xfs/libxfs/xfs_fs.h | 1 + fs/xfs/libxfs/xfs_sb.c | 4 ++++ fs/xfs/xfs_super.c | 4 ++++ 4 files changed, 12 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +3 -1
@@ -373,13 +373,15 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_BIGTIME (1 << 3)/* large timestamps */ #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)/* needs xfs_repair */ #define XFS_SB_FEAT_INCOMPAT_NREXT64 (1 << 5)/* large extent counters */ +#define XFS_SB_FEAT_INCOMPAT_PARENT (1 << 6)/* parent pointers */ #define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ XFS_SB_FEAT_INCOMPAT_META_UUID| \ XFS_SB_FEAT_INCOMPAT_BIGTIME| \ XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \ - XFS_SB_FEAT_INCOMPAT_NREXT64) + XFS_SB_FEAT_INCOMPAT_NREXT64| \ + XFS_SB_FEAT_INCOMPAT_PARENT)#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
fs/xfs/libxfs/xfs_fs.h +1 -0
@@ -237,6 +237,7 @@ typedef struct xfs_fsop_resblks {#define XFS_FSOP_GEOM_FLAGS_BIGTIME (1 << 21)/* 64-bit nsec timestamps */ #define XFS_FSOP_GEOM_FLAGS_INOBTCNT (1 << 22)/* inobt btree counter */ #define XFS_FSOP_GEOM_FLAGS_NREXT64 (1 << 23)/* large extent counters */ +#define XFS_FSOP_GEOM_FLAGS_PARENT (1 << 24)/* parent pointers */ /* * Minimum and maximum sizes need for growth checks.
fs/xfs/libxfs/xfs_sb.c +4 -0
@@ -173,6 +173,8 @@ xfs_sb_version_to_features( features |=XFS_FEAT_NEEDSREPAIR ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_NREXT64 ) features |=XFS_FEAT_NREXT64 ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_PARENT ) + features |=XFS_FEAT_PARENT ;return features; }
@@ -1187,6 +1189,8 @@ xfs_fs_geometry( geo->flags |=XFS_FSOP_GEOM_FLAGS_BIGTIME ;if (xfs_has_inobtcounts (mp)) geo->flags |=XFS_FSOP_GEOM_FLAGS_INOBTCNT ; +if (xfs_has_parent (mp)) + geo->flags |=XFS_FSOP_GEOM_FLAGS_PARENT ;if (xfs_has_sector (mp)) { geo->flags |=XFS_FSOP_GEOM_FLAGS_SECTOR ; geo->logsectsize = sbp->sb_logsectsize;
fs/xfs/xfs_super.c +4 -0
@@ -1655,6 +1655,10 @@ xfs_fs_fill_super(xfs_warn (mp,"EXPERIMENTAL Large extent counts feature in use. Use at your own risk!" ); +if (xfs_has_parent (mp)) +xfs_alert (mp, +"EXPERIMENTAL parent pointer feature enabled. Use at your own risk!" ); + error =xfs_mountfs (mp);if (error) goto out_filestream_unmount;
Signature · 1 line
2.25.1
PATCH RESEND v2 17/18 xfs: Add helper function xfs_attr_list_context_init
In reply to Allison HendersonThis patch adds a helper function xfs_attr_list_context_init used by xfs_attr_list. This function initializes the xfs_attr_list_context structure passed to xfs_attr_list_int. We will need this later to call xfs_attr_list_int_ilocked when the node is already locked.
--- fs/xfs/xfs_file.c | 1 + fs/xfs/xfs_ioctl.c | 54 ++++++++++++++++++++++++++++++++-------------- fs/xfs/xfs_ioctl.h | 2 ++ 3 files changed, 41 insertions(+), 16 deletions(-)
fs/xfs/xfs_file.c +1 -0
@@ -17,6 +17,7 @@#include "xfs_bmap_util.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" #include "xfs_ioctl.h" #include "xfs_trace.h" #include "xfs_log.h"
fs/xfs/xfs_ioctl.c +38 -16
72 lines not on this page · raw message
fs/xfs/xfs_ioctl.h +2 -0
@@ -35,6 +35,8 @@ int xfs_ioc_attrmulti_one(struct file *parfilp, struct inode *inode,int xfs_ioc_attr_list (struct xfs_inode *dp,void __user *ubuf,size_t bufsize,int flags,struct xfs_attrlist_cursor __user *ucursor); +int xfs_ioc_attr_list_context_init (struct xfs_inode *dp,char *buffer, +int bufsize,int flags,struct xfs_attr_list_context *context);extern struct dentry * xfs_handle_to_dentry(
Signature · 1 line
2.25.1
PATCH RESEND v2 18/18 xfs: Add parent pointer ioctl
In reply to Allison HendersonThis patch adds a new file ioctl to retrieve the parent pointer of a given inode
--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_fs.h | 57 ++++++++++++++++ fs/xfs/libxfs/xfs_parent.c | 10 +++ fs/xfs/libxfs/xfs_parent.h | 2 + fs/xfs/xfs_ioctl.c | 95 +++++++++++++++++++++++++- fs/xfs/xfs_ondisk.h | 4 ++ fs/xfs/xfs_parent_utils.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 22 ++++++ 8 files changed, 323 insertions(+), 2 deletions(-)
fs/xfs/Makefile +1 -0
@@ -86,6 +86,7 @@ xfs-y += xfs_aops.o \ xfs_mount.o \ xfs_mru_cache.o \ xfs_pwork.o \ + xfs_parent_utils.o \ xfs_reflink.o \ xfs_stats.o \ xfs_super.o \
fs/xfs/libxfs/xfs_fs.h +57 -0
75 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.c +10 -0
@@ -26,6 +26,16 @@#include "xfs_xattr.h" #include "xfs_parent.h" +/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */ +void +xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec) +{ + xpp->xpp_ino =be64_to_cpu (rec->p_ino); + xpp->xpp_gen =be32_to_cpu (rec->p_gen); + xpp->xpp_diroffset =be32_to_cpu (rec->p_diroffset); +} +/* * Parent pointer attribute handling. *
fs/xfs/libxfs/xfs_parent.h +2 -0
@@ -23,6 +23,8 @@ void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec, uint32_t p_diroffset);void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec,struct xfs_parent_name_rec *rec); +void xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec);int xfs_parent_init (xfs_mount_t *mp,xfs_inode_t *ip,struct xfs_name *target_name,struct xfs_parent_defer **parentp);
fs/xfs/xfs_ioctl.c +93 -2
125 lines not on this page · raw message
fs/xfs/xfs_ondisk.h +4 -0
@@ -135,6 +135,10 @@ xfs_check_ondisk_structs(void) XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 40); XFS_CHECK_STRUCT_SIZE(struct xfs_attrd_log_format, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr, 280); + XFS_CHECK_STRUCT_SIZE(struct xfs_pptr_info, 104); +/* * The v5 superblock format extended several v4 header structures with * additional data. While new fields are only accessible on v5
fs/xfs/xfs_parent_utils.c new file +134 -0
134 lines not on this page · raw message
fs/xfs/xfs_parent_utils.h new file +22 -0
@@ -0,0 +1,22 @@ +/* +* Copyright (c) 2017 Oracle, Inc. +* All Rights Reserved. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write the Free Software Foundation Inc. +*/ +#ifndef __XFS_PARENT_UTILS_H__ +#define __XFS_PARENT_UTILS_H__ + +int xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi); +#endif /* __XFS_PARENT_UTILS_H__ */
Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 18/18] xfs: Add parent pointer ioctl
Quoted 32 lines · Allison Henderson
This patch adds a new file ioctl to retrieve the parent pointer of a given inode
Signed-off-by Allison Henderson <[email protected]>--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_fs.h | 57 ++++++++++++++++ fs/xfs/libxfs/xfs_parent.c | 10 +++ fs/xfs/libxfs/xfs_parent.h | 2 + fs/xfs/xfs_ioctl.c | 95 +++++++++++++++++++++++++- fs/xfs/xfs_ondisk.h | 4 ++ fs/xfs/xfs_parent_utils.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 22 ++++++ 8 files changed, 323 insertions(+), 2 deletions(-)fs/xfs/Makefile +1
-0@@ -86,6 +86,7 @@ xfs-y += xfs_aops.o \ xfs_mount.o \ xfs_mru_cache.o \ xfs_pwork.o \ + xfs_parent_utils.o \ xfs_reflink.o \ xfs_stats.o \ xfs_super.o \fs/xfs/libxfs/xfs_fs.h +0
-0
fs/xfs/libxfs/xfs_fs.h typedef struct xfs_fsop_handlereq { · line 577
Quoted 4 lines
fs/xfs/libxfs/xfs_fs.h +1
-0@@ -574,6 +574,7 @@ typedef struct xfs_fsop_handlereq {#define XFS_IOC_ATTR_SECURE 0x0008/* use attrs in security namespace */ #define XFS_IOC_ATTR_CREATE 0x0010/* fail if attr already exists */ #define XFS_IOC_ATTR_REPLACE 0x0020/* fail if attr does not exist */ +#define XFS_IOC_ATTR_PARENT 0x0040/* use attrs in parent namespace */
This is the userspace API header, so I wonder -- should we allow XFS_IOC_ATTRLIST_BY_HANDLE and XFS_IOC_ATTRMULTI_BY_HANDLE to access parent pointers?
I think it's *definitely* incorrect to let ATTR_OP_REMOVE or ATTR_OP_SET (attrmulti subcommands) to mess with parent pointers.
I don't think attrlist or ATTR_OP_GET should be touching them either, particularly since you're defining a new ioctl to extract *only* the parent pointers.
If there wasn't XFS_IOC_GETPPOINTER then perhaps it would be ok to allow reads via ATTRLIST/ATTRMULTI. But even then, I don't think we want things like xfsdump to think that it has to preserve those attributes since xfsrestore will reconstruct the directory tree (and hence the pptrs) for us.
Quoted 16 lines
fs/xfs/libxfs/xfs_fs.h +9
-0typedef struct xfs_attrlist_cursor { __u32 opaque[4];@@ -752,6 +753,61 @@ struct xfs_scrub_metadata { XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED)#define XFS_SCRUB_FLAGS_ALL (XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT) +#define XFS_PPTR_MAXNAMELEN 256 + +/* return parents of the handle, not the open fd */ +#define XFS_PPTR_IFLAG_HANDLE (1U << 0) + +/* target was the root directory */ +#define XFS_PPTR_OFLAG_ROOT (1U << 1) + +/* Cursor is done iterating pptrs */
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 774
Quoted 10 lines
fs/xfs/libxfs/xfs_fs.h +10
-0+#define XFS_PPTR_OFLAG_DONE (1U << 2) + +/* Get an inode parent pointer through ioctl */ +struct xfs_parent_ptr { +__u64 xpp_ino;/* Inode */ +__u32 xpp_gen;/* Inode generation */ +__u32 xpp_diroffset;/* Directory offset */ +__u32 xpp_namelen;/* File name length */ +__u32 xpp_pad; +__u8 xpp_name[XFS_PPTR_MAXNAMELEN ];/* File name */
Since xpp_name is a fixed-length array that is long enough to ensure that there's a null at the end of the name, we don't need xpp_namelen.
I wonder if xpp_namelen and xpp_pad should simply turn into a u64 field that's defined zero for future expansion?
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 783
Quoted 9 lines
fs/xfs/libxfs/xfs_fs.h +9
-0+}; + +/* Iterate through an inodes parent pointers */ +struct xfs_pptr_info { +struct xfs_handle pi_handle; +struct xfs_attrlist_cursor pi_cursor; +__u32 pi_flags; +__u32 pi_reserved; +__u32 pi_ptrs_size;
Is this the number of elements in pi_parents[]?
Quoted 14 lines
fs/xfs/libxfs/xfs_fs.h +14
-0+__u32 pi_ptrs_used; +__u64 pi_reserved2[6 ]; + +/* +* An array of struct xfs_parent_ptr follows the header +* information. Use XFS_PPINFO_TO_PP() to access the +* parent pointer array entries. +*/ +struct xfs_parent_ptr pi_parents[]; +}; + +static inline size_t +xfs_pptr_info_sizeof (int nr_ptrs) +{
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 807
Quoted 10 lines
fs/xfs/libxfs/xfs_fs.h +10
-0+return sizeof (struct xfs_pptr_info ) + + (nr_ptrs *sizeof (struct xfs_parent_ptr )); +} + +static inline struct xfs_parent_ptr * +xfs_ppinfo_to_pp ( +struct xfs_pptr_info *info, +int idx) +{ +
Nit: extra space.
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 856
Quoted 11 lines
fs/xfs/libxfs/xfs_fs.h +4
-0+return &info->pi_parents[idx]; +} +/* * ioctl limits */ @@ -797,6 +853,7 @@ struct xfs_scrub_metadata {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +#define XFS_IOC_GETPPOINTER _IOR ('X', 62, struct xfs_parent_ptr)
I wonder if this name should more strongly emphasize that it's for reading the parents of a file?
#define XFS_IOC_GETPARENTS _IOWR(...)
Also, the ioctl reads and writes its parameter, so this is _IOWR, not _IOR.
BTW, is there a sample manpage somewhere?
Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h +0
-0/* * ioctl commands that replace IRIX syssgi()'s fs/xfs/libxfs/xfs_parent.c +0
-0
fs/xfs/libxfs/xfs_parent.c line 32
Quoted 7 lines
fs/xfs/libxfs/xfs_parent.c +4
-0@@ -26,6 +26,16 @@#include "xfs_xattr.h" #include "xfs_parent.h" +/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */ +void +xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec)
The second parameter ought to be const struct xfs_parent_name_rec *rec to make it unambiguous to readers which is the source and which is the destination argument.
Quoted 44 lines
fs/xfs/libxfs/xfs_parent.c +6
-0+{ + xpp->xpp_ino =be64_to_cpu (rec->p_ino); + xpp->xpp_gen =be32_to_cpu (rec->p_gen); + xpp->xpp_diroffset =be32_to_cpu (rec->p_diroffset); +} +/* * Parent pointer attribute handling. * fs/xfs/libxfs/xfs_parent.h +2
-0@@ -23,6 +23,8 @@ void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec, uint32_t p_diroffset);void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec,struct xfs_parent_name_rec *rec); +void xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec);int xfs_parent_init (xfs_mount_t *mp,xfs_inode_t *ip,struct xfs_name *target_name,struct xfs_parent_defer **parentp);fs/xfs/xfs_ioctl.c +3
-0@@ -37,6 +37,7 @@#include "xfs_health.h" #include "xfs_reflink.h" #include "xfs_ioctl.h" +#include "xfs_parent_utils.h" #include "xfs_xattr.h" #include <linux/mount.h> @@ -355,6 +356,8 @@ xfs_attr_filter(return XFS_ATTR_ROOT ;if (ioc_flags &XFS_IOC_ATTR_SECURE )return XFS_ATTR_SECURE ; +if (ioc_flags &XFS_IOC_ATTR_PARENT ) +return XFS_ATTR_PARENT ;return 0 ; }
fs/xfs/xfs_ioctl.c xfs_ioc_attr_list( · line 430
Quoted 7 lines
fs/xfs/xfs_ioctl.c +2
-1@@ -422,7 +425,8 @@ xfs_ioc_attr_list(/* * Reject flags, only allow namespaces. */ -if (flags & ~(XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE )) +if (flags & ~(XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE | +XFS_IOC_ATTR_PARENT ))return -EINVAL ;
I think xfs_ioc_attrmulti_one needs filtering for XFS_IOC_ATTR_PARENT, if we're still going to allow attrlist/attrmulti to return parent pointers.
Quoted 22 lines
fs/xfs/xfs_ioctl.c +16
-0if (flags == (XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE ))return -EINVAL ;@@ -1679,6 +1683,92 @@ xfs_ioc_scrub_metadata(return 0 ; } +/* +* IOCTL routine to get the parent pointers of an inode and return it to user +* space. Caller must pass a buffer space containing a struct xfs_pptr_info, +* followed by a region large enough to contain an array of struct +* xfs_parent_ptr of a size specified in pi_ptrs_size. If the inode contains +* more parent pointers than can fit in the buffer space, caller may re-call +* the function using the returned pi_cursor to resume iteration. The +* number of xfs_parent_ptr returned will be stored in pi_ptrs_used. +* +* Returns 0 on success or non-zero on failure +*/ +STATIC int +xfs_ioc_get_parent_pointer ( +struct file *filp, +void __user *arg) +{
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1711
Quoted 10 lines
fs/xfs/xfs_ioctl.c +10
-0+struct xfs_pptr_info *ppi =NULL ; +int error =0 ; +struct xfs_inode *ip =XFS_I (file_inode (filp)); +struct xfs_mount *mp = ip->i_mount; + +if (!capable (CAP_SYS_ADMIN )) +return -EPERM ; + +/* Allocate an xfs_pptr_info to put the user data */ + ppi =kmem_alloc (sizeof (struct xfs_pptr_info ),0 );
New code should call kmalloc instead of the old kmem_alloc wrapper.
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1716
Quoted 5 lines
fs/xfs/xfs_ioctl.c +5
-0+if (!ppi) +return -ENOMEM ; + +/* Copy the data from the user */ + error =copy_from_user (ppi, arg,sizeof (struct xfs_pptr_info ));
Note: copy_from_user returns the number of bytes *not* copied. If you receive a nonzero return value, error usually gets set to EFAULT.
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1726
Quoted 10 lines
fs/xfs/xfs_ioctl.c +10
-0+if (error) + goto out; + +/* Check size of buffer requested by user */ +if (xfs_pptr_info_sizeof (ppi->pi_ptrs_size) >XFS_XATTR_LIST_MAX ) { + error = -ENOMEM ; + goto out; + } + +if (ppi->pi_flags !=0 && ppi->pi_flags !=XFS_PPTR_IFLAG_HANDLE ) {
if (ppi->pi_flags & ~XFS_PPTR_IFLAG_HANDLE) ?
(If we really want to be pedantic, this really ought to be:
#define XFS_PPTR_IFLAG_ALL (XFS_PPTR_IFLAG_HANDLE)
if (ppi->pi_flags & ~XFS_PPTR_IFLAG_ALL) return -EINVAL;
Or you could be more flexible, since the kernel could just set the OFLAGs appropriately and not care about their value on input:
#define XFS_PPTR_FLAG_ALL (XFS_PPTR_IFLAG_HANDLE | XFS_PPTR_OFLAG...)
if (ppi->pi_flags & ~XFS_PPTR_FLAG_ALL) return -EINVAL; ppi->pi_flags &= ~(XFS_PPTR_OFLAG_ROOT | XFS_PPTR_OFLAG_DONE);
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1738
Quoted 12 lines
fs/xfs/xfs_ioctl.c +12
-0+ error = -EINVAL ; + goto out; + } + +/* +* Now that we know how big the trailing buffer is, expand +* our kernel xfs_pptr_info to be the same size +*/ + ppi =krealloc (ppi,xfs_pptr_info_sizeof (ppi->pi_ptrs_size), +GFP_NOFS | __GFP_NOFAIL); +if (!ppi) +return -ENOMEM ;
Why NOFS and NOFAIL? We don't have any writeback resources locked (transactions and ILOCKs) so we can hit ourselves up for memory.
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1740
Quoted 2 lines
fs/xfs/xfs_ioctl.c +2
-0+ +if (ppi->pi_flags ==XFS_PPTR_IFLAG_HANDLE ) {
if (ppi->pi_flags & XFS_PPTR_IFLAG_HANDLE) {Quoted 13 lines
fs/xfs/xfs_ioctl.c +13
-0+ error =xfs_iget (mp,NULL , ppi->pi_handle.ha_fid.fid_ino, +0 ,0 , &ip); +if (error) + goto out; + +if (VFS_I (ip)->i_generation != ppi->pi_handle.ha_fid.fid_gen) { + error = -EINVAL ; + goto out; + } + } + +if (ip->i_ino == mp->m_sb.sb_rootino) + ppi->pi_flags |=XFS_PPTR_OFLAG_ROOT ;
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1763
Quoted 10 lines
fs/xfs/xfs_ioctl.c +10
-0+ +/* Get the parent pointers */ + error =xfs_attr_get_parent_pointer (ip, ppi); + +if (error) + goto out; + +/* Copy the parent pointers back to the user */ + error =copy_to_user (arg, ppi, +xfs_pptr_info_sizeof (ppi->pi_ptrs_size));
Same note as the one I made for copy_from_user.
Quoted 48 lines
fs/xfs/xfs_ioctl.c +10
-1+if (error) + goto out; + +out: +kmem_free (ppi); +return error; +} +int xfs_ioc_swapext (xfs_swapext_t *sxp)@@ -1968,7 +2058,8 @@ xfs_file_ioctl(case XFS_IOC_FSGETXATTRA :return xfs_ioc_fsgetxattra (ip, arg); - +case XFS_IOC_GETPPOINTER : +return xfs_ioc_get_parent_pointer (filp, arg);case XFS_IOC_GETBMAP :case XFS_IOC_GETBMAPA :case XFS_IOC_GETBMAPX :fs/xfs/xfs_ondisk.h +4
-0@@ -135,6 +135,10 @@ xfs_check_ondisk_structs(void) XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 40); XFS_CHECK_STRUCT_SIZE(struct xfs_attrd_log_format, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr, 280); + XFS_CHECK_STRUCT_SIZE(struct xfs_pptr_info, 104); +/* * The v5 superblock format extended several v4 header structures with * additional data. While new fields are only accessible on v5 fs/xfs/xfs_parent_utils.c new file +6
-0@@ -0,0 +1,134 @@ +/* +* Copyright (c) 2015 Red Hat, Inc. +* All rights reserved. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as
fs/xfs/xfs_parent_utils.c line 16
Quoted 10 lines
fs/xfs/xfs_parent_utils.c new file +10
-0+* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write the Free Software Foundation +*/
Please condense this boilerplate down to a SPDX tag and a copyright statement.
Quoted 21 lines
fs/xfs/xfs_parent_utils.c new file +21
-0+#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_bmap_btree.h" +#include "xfs_inode.h" +#include "xfs_error.h" +#include "xfs_trace.h" +#include "xfs_trans.h" +#include "xfs_da_format.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" +#include "xfs_ioctl.h" +#include "xfs_parent.h" +#include "xfs_da_btree.h" + +/* +* Get the parent pointers for a given inode
fs/xfs/xfs_parent_utils.c line 47
Quoted 10 lines
fs/xfs/xfs_parent_utils.c new file +10
-0+* +* Returns 0 on success and non zero on error +*/ +int +xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi) + +{ + +struct xfs_attrlist *alist;
int
xfs_attr_get_parent_pointer(
struct xfs_inode *ip,
struct xfs_pptr_info *ppi)
{
struct xfs_attrlist *alist;Quoted 8 lines
fs/xfs/xfs_parent_utils.c new file +8
-0+struct xfs_attrlist_ent *aent; +struct xfs_parent_ptr *xpp; +struct xfs_parent_name_rec *xpnr; +char *namebuf; +unsigned int namebuf_size; +int name_len; +int error =0 ; +unsigned int ioc_flags =XFS_IOC_ATTR_PARENT ;
fs/xfs/xfs_parent_utils.c line 65
Quoted 10 lines
fs/xfs/xfs_parent_utils.c new file +10
-0+unsigned int flags =XFS_ATTR_PARENT ; +int i; +struct xfs_attr_list_context context; + +/* Allocate a buffer to store the attribute names */ + namebuf_size =sizeof (struct xfs_attrlist ) + + (ppi->pi_ptrs_size) *sizeof (struct xfs_attrlist_ent ); + namebuf =kvzalloc (namebuf_size,GFP_KERNEL ); +if (!namebuf) +return -ENOMEM ;
Do we need the buffer to be zeroed if xfs_attr_list is just going to set its contents?
fs/xfs/xfs_parent_utils.c line 69
Quoted 4 lines
fs/xfs/xfs_parent_utils.c new file +4
-0+ +memset (&context,0 ,sizeof (struct xfs_attr_list_context )); + error =xfs_ioc_attr_list_context_init (ip, namebuf, namebuf_size, + ioc_flags, &context);
Aha, so the internal implementation has access to xfs_attr_list_context before it calls into the attr list code. Ok, in that case, xfs_fs.h doesn't need the XFS_IOC_ATTR_PARENT flag, and you can set context.attr_filter = XFS_ATTR_PARENT here. Then we don't have to worry about the existing xattr bulk ioctls returning parent pointers.
fs/xfs/xfs_parent_utils.c line 76
Quoted 7 lines
fs/xfs/xfs_parent_utils.c new file +7
-0+ +/* Copy the cursor provided by caller */ +memcpy (&context.cursor, &ppi->pi_cursor, +sizeof (struct xfs_attrlist_cursor )); + +if (error) + goto out_kfree;
Why does the error check come after copying the cursor into the onstack variable?
fs/xfs/xfs_parent_utils.c line 78
Quoted 2 lines
fs/xfs/xfs_parent_utils.c new file +2
-0+ +xfs_ilock (ip,XFS_ILOCK_EXCL );
xfs_ilock_attr_map_shared() ?
Quoted 15 lines
fs/xfs/xfs_parent_utils.c new file +15
-0+ + error =xfs_attr_list_ilocked (&context); +if (error) + goto out_kfree; + + alist = (struct xfs_attrlist *)namebuf; +for (i =0 ; i < alist->al_count; i++) { +struct xfs_da_args args = { + .geo = ip->i_mount->m_attr_geo, + .whichfork =XFS_ATTR_FORK , + .dp = ip, + .namelen =sizeof (struct xfs_parent_name_rec ), + .attr_filter = flags, + .op_flags =XFS_DA_OP_OKNOENT , + };
fs/xfs/xfs_parent_utils.c line 103
Quoted 10 lines
fs/xfs/xfs_parent_utils.c new file +10
-0+ + xpp =xfs_ppinfo_to_pp (ppi, i); +memset (xpp,0 ,sizeof (struct xfs_parent_ptr )); + aent = (struct xfs_attrlist_ent *) + &namebuf[alist->al_offset[i]]; + xpnr = (struct xfs_parent_name_rec *)(aent->a_name); + +if (aent->a_valuelen >XFS_PPTR_MAXNAMELEN ) { + error = -ERANGE ; + goto out_kfree;
If a parent pointer has a name longer than MAXNAMELEN then isn't that a corruption? And in that case, -EFSCORRUPTED would be more appropriate here, right?
fs/xfs/xfs_parent_utils.c line 112
Quoted 9 lines
fs/xfs/xfs_parent_utils.c new file +9
-0+ } + name_len = aent->a_valuelen; + + args.name = (char *)xpnr; + args.hashval =xfs_da_hashname (args.name, args.namelen), + args.value = (unsigned char *)(xpp->xpp_name); + args.valuelen = name_len; + + error =xfs_attr_get_ilocked (&args);
If error is ENOENT (or ENOATTR or whatever the return value is when the attr doesn't exist) then shouldn't that be treated as a corruption too? We still hold the ILOCK from earlier. I don't think OKNOENT is correct either.
fs/xfs/xfs_parent_utils.c line 118
Quoted 6 lines
fs/xfs/xfs_parent_utils.c new file +6
-0+ error = (error == -EEXIST ?0 : error); +if (error) + goto out_kfree; + + xpp->xpp_namelen = name_len; +xfs_init_parent_ptr (xpp, xpnr);
Also, should we validate xpnr before copying it out to userspace? If, say, the inode number is bogus, that should generate an EFSCORRUPTED.
fs/xfs/xfs_parent_utils.c line 126
Quoted 8 lines
fs/xfs/xfs_parent_utils.c new file +8
-0+ } + ppi->pi_ptrs_used = alist->al_count; +if (!alist->al_more) + ppi->pi_flags |=XFS_PPTR_OFLAG_DONE ; + +/* Update the caller with the current cursor position */ +memcpy (&ppi->pi_cursor, &context.cursor, +sizeof (struct xfs_attrlist_cursor ));
Glad you remembered to do this; attrmulti forgot to do this for a long time. :)
fs/xfs/xfs_parent_utils.c line 130
Quoted 4 lines
fs/xfs/xfs_parent_utils.c new file +4
-0+ +out_kfree: +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +kmem_free (namebuf);
kvfree, since you got namebuf from kvzalloc.
fs/xfs/xfs_parent_utils.h line 2
Quoted 12 lines
fs/xfs/xfs_parent_utils.c new file +4
-0+ +return error; +} +fs/xfs/xfs_parent_utils.h new file +2
-0@@ -0,0 +1,22 @@ +/* +* Copyright (c) 2017 Oracle, Inc.
2022?
fs/xfs/xfs_parent_utils.h line 15
Quoted 13 lines
fs/xfs/xfs_parent_utils.h new file +13
-0+* All Rights Reserved. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write the Free Software Foundation Inc.
This also needs to be condensed to a SPDX header and a copyright statement.
Quoted 10 lines
fs/xfs/xfs_parent_utils.h new file +7
-0+*/ +#ifndef __XFS_PARENT_UTILS_H__ +#define __XFS_PARENT_UTILS_H__ + +int xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi); +#endif /* __XFS_PARENT_UTILS_H__ */ Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 18/18] xfs: Add parent pointer ioctl
Quoted 39 lines · Darrick J. Wong
Quoted 38 lines · Allison Henderson
This patch adds a new file ioctl to retrieve the parent pointer of a
given inodeSigned-off-by Allison Henderson <[email protected]>--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_fs.h | 57 ++++++++++++++++ fs/xfs/libxfs/xfs_parent.c | 10 +++ fs/xfs/libxfs/xfs_parent.h | 2 + fs/xfs/xfs_ioctl.c | 95 +++++++++++++++++++++++++- fs/xfs/xfs_ondisk.h | 4 ++ fs/xfs/xfs_parent_utils.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 22 ++++++ 8 files changed, 323 insertions(+), 2 deletions(-)fs/xfs/Makefile +1
-0@@ -86,6 +86,7 @@ xfs-y += xfs_aops.o \ xfs_mount.o \ xfs_mru_cache.o \ xfs_pwork.o \ + xfs_parent_utils.o \ xfs_reflink.o \ xfs_stats.o \ xfs_super.o \fs/xfs/libxfs/xfs_fs.h +0
-0@@ -574,6 +574,7 @@ typedef struct xfs_fsop_handlereq {#define XFS_IOC_ATTR_SECURE 0x0008/* use attrs in security namespace */
#defineXFS_IOC_ATTR_CREATE0x0010/* fail if attr
already exists */
Quoted 8 lines
Quoted 4 lines
#define
XFS_IOC_ATTR_REPLACE0x0020/* fail if attr
does not exist */
+#defineXFS_IOC_ATTR_PARENT0x0040/* use attrs in parent
namespace */This is the userspace API header, so I wonder -- should we allow
XFS_IOC_ATTRLIST_BY_HANDLEandXFS_IOC_ATTRMULTI_BY_HANDLEto access parent pointers?
Well, the ioc is how the test cases get the pptrs back out in order to verify parent pointers are working. So we need to keep at least that, but then I think it makes worrying about other forms of access feel sort of silly since we're not really hiding anything. They would have to pass in the parent filter flag which wasnt allowable until now, so it's not like having pptrs appear in the list when asked for is inappropriate.
Quoted 4 lines
I think it's *definitely* incorrect to let
ATTR_OP_REMOVEorATTR_OP_SET
(attrmulti subcommands) to mess with parent pointers.
Ok, I can see if I can add some sanity checking there.
Quoted 11 lines
I don't think attrlist or
ATTR_OP_GETshould be touching them either, particularly since you're defining a new ioctl to extract *only* the parent pointers.If there wasn't
XFS_IOC_GETPPOINTERthen perhaps it would be ok to allow
reads via ATTRLIST/ATTRMULTI. But even then, I don't think we want things like xfsdump to think that it has to preserve those attributes since xfsrestore will reconstruct the directory tree (and hence the pptrs) for us.
Hrmm, not sure I follow this part, the point of pptrs are to reconstruct the tree, so wouldnt we want them preserved?
Quoted 37 lines
Quoted 31 lines
typedef struct
xfs_attrlist_cursor{fs/xfs/libxfs/xfs_fs.h +15
-0__u32 opaque[4 ];@@ -752,6 +753,61 @@ struct xfs_scrub_metadata { XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED)#define XFS_SCRUB_FLAGS_ALL (XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT) +#define XFS_PPTR_MAXNAMELEN 256 + +/* return parents of the handle, not the open fd */ +#define XFS_PPTR_IFLAG_HANDLE (1U << 0) + +/* target was the root directory */ +#define XFS_PPTR_OFLAG_ROOT (1U << 1) + +/* Cursor is done iterating pptrs */ +#define XFS_PPTR_OFLAG_DONE (1U << 2) + +/* Get an inode parent pointer through ioctl */ +struct xfs_parent_ptr { +__u64 xpp_ino;/* Inode */ +__u32 xpp_gen;/* Inode generation */
fs/xfs/libxfs/xfs_fs.h +1
-0+__u32 xpp_diroffset;/* Directory offset */
fs/xfs/libxfs/xfs_fs.h +1
-0+__u32 xpp_namelen;/* File name length */
fs/xfs/libxfs/xfs_fs.h +2
-0+__u32 xpp_pad; +__u8 xpp_name[XFS_PPTR_MAXNAMELEN ];/* File Quoted 1 line
name */
Since
xpp_nameis a fixed-length array that is long enough to ensure that there's a null at the end of the name, we don't needxpp_namelen.
Quoted 3 lines
I wonder if
xpp_namelenandxpp_padshould simply turn into a u64 field
that's defined zero for future expansion?
Sure, I'll see if I can remove it and add a reserved field
Quoted 12 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 783
Quoted 10 lines
fs/xfs/libxfs/xfs_fs.h +9
-0+}; + +/* Iterate through an inodes parent pointers */ +struct xfs_pptr_info { +struct xfs_handle pi_handle; +struct xfs_attrlist_cursor pi_cursor; +__u32 pi_flags; +__u32 pi_reserved; +__u32 pi_ptrs_size;Is this the number of elements in pi_parents[]?
Yes, it's the number parent pointers in the array
Quoted 19 lines
Quoted 19 lines
fs/xfs/libxfs/xfs_fs.h +18
-0+__u32 pi_ptrs_used; +__u64 pi_reserved2[6 ]; + +/* +* An array of struct xfs_parent_ptr follows the header +* information. Use XFS_PPINFO_TO_PP() to access the +* parent pointer array entries. +*/ +struct xfs_parent_ptr pi_parents[]; +}; + +static inline size_t +xfs_pptr_info_sizeof (int nr_ptrs) +{ +return sizeof (struct xfs_pptr_info ) + + (nr_ptrs *sizeof (struct xfs_parent_ptr )); +} +
Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 807
Quoted 6 lines
fs/xfs/libxfs/xfs_fs.h +6
-0+static inline struct xfs_parent_ptr * +xfs_ppinfo_to_pp ( +struct xfs_pptr_info *info, +int idx) +{ +Nit: extra space.
Will fix
Quoted 20 lines
Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h +3
-0+return &info->pi_parents[idx]; +} +/* * ioctl limits */ fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 859
Quoted 7 lines
fs/xfs/libxfs/xfs_fs.h +1
-0@@ -797,6 +853,7 @@ struct xfs_scrub_metadata {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +#define XFS_IOC_GETPPOINTER _IOR ('X', 62, struct xfs_parent_ptr)I wonder if this name should more strongly emphasize that it's for reading
the parents of a file?
Quoted 1 line
#define
XFS_IOC_GETPARENTS_IOWR(...)
Sure, that sounds fine i think
Quoted 5 lines
Also, the ioctl reads and writes its parameter, so this is
_IOWR, not_IOR.BTW, is there a sample manpage somewhere?
The userspace branch adds some new flags to xfsprogs and some usage help to explain how to use them. See the last patch in the branch:
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrsv2
But it's just for printing the parent pointers out, it doesn't have a man page for how to write your own ioctl. I suppose we could add it though.
Quoted 13 lines
Quoted 13 lines
/*
* ioctl commands that replace IRIX syssgi()'sfs/xfs/libxfs/xfs_parent.c +0
-0@@ -26,6 +26,16 @@#include "xfs_xattr.h" #include "xfs_parent.h"
Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c line 32
Quoted 4 lines
fs/xfs/libxfs/xfs_parent.c +4
-0+/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */ +void +xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec)The second parameter ought to be const struct
xfs_parent_name_rec*rec
to make it unambiguous to readers which is the source and which is the
destination argument.
Ok, will update
Quoted 46 lines
Quoted 46 lines
fs/xfs/libxfs/xfs_parent.c +6
-0+{ + xpp->xpp_ino =be64_to_cpu (rec->p_ino); + xpp->xpp_gen =be32_to_cpu (rec->p_gen); + xpp->xpp_diroffset =be32_to_cpu (rec->p_diroffset); +} +/* * Parent pointer attribute handling. * fs/xfs/libxfs/xfs_parent.h +2
-0@@ -23,6 +23,8 @@ void xfs_init_parent_name_rec(structxfs_parent_name_rec *rec, uint32_t p_diroffset);void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec,struct xfs_parent_name_rec *rec); +void xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec); int xfs_parent_init(xfs_mount_t *mp, xfs_inode_t *ip, struct xfs_name *target_name,fs/xfs/libxfs/xfs_parent.h +0
-0struct xfs_parent_defer **parentp);fs/xfs/xfs_ioctl.c +3
-0@@ -37,6 +37,7 @@#include "xfs_health.h" #include "xfs_reflink.h" #include "xfs_ioctl.h" +#include "xfs_parent_utils.h" #include "xfs_xattr.h" #include <linux/mount.h> @@ -355,6 +356,8 @@ xfs_attr_filter(return XFS_ATTR_ROOT ;if (ioc_flags &XFS_IOC_ATTR_SECURE )return XFS_ATTR_SECURE ; +if (ioc_flags &XFS_IOC_ATTR_PARENT ) +return XFS_ATTR_PARENT ;return 0 ; }
Quoted 13 lines
fs/xfs/xfs_ioctl.c xfs_ioc_attr_list( · line 430
Quoted 8 lines
fs/xfs/xfs_ioctl.c +2
-1@@ -422,7 +425,8 @@ xfs_ioc_attr_list(/* * Reject flags, only allow namespaces. */ -if (flags & ~(XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE )) +if (flags & ~(XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE | +XFS_IOC_ATTR_PARENT ))return -EINVAL ;I think
xfs_ioc_attrmulti_oneneeds filtering forXFS_IOC_ATTR_PARENT,
if we're still going to allow attrlist/attrmulti to return parent pointers.
Ok, will update that one as well then
Quoted 33 lines
Quoted 33 lines
fs/xfs/xfs_ioctl.c +2
-0if (flags == (XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE ))return -EINVAL ;@@ -1679,6 +1683,92 @@ xfs_ioc_scrub_metadata(return 0 ; } +/* +* IOCTL routine to get the parent pointers of an inode and return it to user
+ * space. Caller must pass a buffer space containing a structxfs_pptr_info,
+ * followed by a region large enough to contain an array of struct
+ *xfs_parent_ptrof a size specified inpi_ptrs_size. If the inode contains
+ * more parent pointers than can fit in the buffer space, caller may re-call
+ * the function using the returnedpi_cursorto resume iteration. The
+ * number ofxfs_parent_ptrreturned will be stored inpi_ptrs_used.
+ *
+ * Returns 0 on success or non-zero on failure
+ */fs/xfs/xfs_ioctl.c +9
-0+STATIC int +xfs_ioc_get_parent_pointer ( +struct file *filp, +void __user *arg) +{ +struct xfs_pptr_info *ppi =NULL ; +int error =0 ; +struct xfs_inode *ip =XFS_I (file_inode (filp)); +struct xfs_mount *mp = ip->i_mount;
Quoted 9 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1711
Quoted 6 lines
fs/xfs/xfs_ioctl.c +6
-0+ +if (!capable (CAP_SYS_ADMIN )) +return -EPERM ; + +/* Allocate an xfs_pptr_info to put the user data */ + ppi =kmem_alloc (sizeof (struct xfs_pptr_info ),0 );New code should call kmalloc instead of the old
kmem_allocwrapper.
Ok, will update
Quoted 9 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1716
Quoted 5 lines
fs/xfs/xfs_ioctl.c +5
-0+if (!ppi) +return -ENOMEM ; + +/* Copy the data from the user */ + error =copy_from_user (ppi, arg,sizeof (struct xfs_pptr_info ));Note:
copy_from_userreturns the number of bytes *not* copied. If you
receive a nonzero return value, error usually gets set to EFAULT.
ooooh. ok, will fix that then.
Quoted 26 lines
Quoted 12 lines
fs/xfs/xfs_ioctl.c +5
-0+if (error) + goto out; + +/* Check size of buffer requested by user */ + if (xfs_pptr_info_sizeof(ppi->pi_ptrs_size) >
XFS_XATTR_LIST_MAX) {fs/xfs/xfs_ioctl.c +5
-0+ error = -ENOMEM ; + goto out; + } + + if (ppi->pi_flags != 0 && ppi->pi_flags !=Quoted 1 line
XFS_PPTR_IFLAG_HANDLE) {if (ppi->pi_flags & ~XFS_PPTR_IFLAG_HANDLE) ?(If we really want to be pedantic, this really ought to be:
#define
XFS_PPTR_IFLAG_ALL(XFS_PPTR_IFLAG_HANDLE)if (ppi->pi_flags & ~XFS_PPTR_IFLAG_ALL) return -EINVAL;Or you could be more flexible, since the kernel could just set the OFLAGs appropriately and not care about their value on input:
Quoted 7 lines
#define
XFS_PPTR_FLAG_ALL(XFS_PPTR_IFLAG_HANDLE|XFS_PPTR_OFLAG...)if (ppi->pi_flags & ~XFS_PPTR_FLAG_ALL) return -EINVAL; ppi->pi_flags &= ~(XFS_PPTR_OFLAG_ROOT | XFS_PPTR_OFLAG_DONE);
Oh, I see, sure that makes sense.
Quoted 16 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1738
Quoted 13 lines
fs/xfs/xfs_ioctl.c +12
-0+ error = -EINVAL ; + goto out; + } + +/* +* Now that we know how big the trailing buffer is, expand +* our kernel xfs_pptr_info to be the same size +*/ + ppi =krealloc (ppi,xfs_pptr_info_sizeof (ppi->pi_ptrs_size), +GFP_NOFS | __GFP_NOFAIL); +if (!ppi) +return -ENOMEM ;Why NOFS and NOFAIL? We don't have any writeback resources locked (transactions and ILOCKs) so we can hit ourselves up for memory.
Ok, will update
Quoted 5 lines
fs/xfs/xfs_ioctl.c
Quoted 3 lines
fs/xfs/xfs_ioctl.c +2
-0+ +if (ppi->pi_flags ==XFS_PPTR_IFLAG_HANDLE ) {if (ppi->pi_flags & XFS_PPTR_IFLAG_HANDLE) {
ok, will fix
Quoted 20 lines
Quoted 20 lines
fs/xfs/xfs_ioctl.c +1
-0+ error = xfs_iget(mp, NULL, ppi-Quoted 1 line
pi_handle.ha_fid.fid_ino,
fs/xfs/xfs_ioctl.c +5
-0+ 0, 0, &ip); +if (error) + goto out; + + if (VFS_I(ip)->i_generation != ppi-Quoted 1 line
pi_handle.ha_fid.fid_gen) {
fs/xfs/xfs_ioctl.c +11
-0+ error = -EINVAL ; + goto out; + } + } + +if (ip->i_ino == mp->m_sb.sb_rootino) + ppi->pi_flags |=XFS_PPTR_OFLAG_ROOT ; + +/* Get the parent pointers */ + error =xfs_attr_get_parent_pointer (ip, ppi); +
Quoted 9 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1763
Quoted 6 lines
fs/xfs/xfs_ioctl.c +6
-0+if (error) + goto out; + +/* Copy the parent pointers back to the user */ + error =copy_to_user (arg, ppi, +xfs_pptr_info_sizeof (ppi->pi_ptrs_size));Same note as the one I made for
copy_from_user.
Will update
Quoted 57 lines
Quoted 57 lines
fs/xfs/xfs_ioctl.c +10
-1+if (error) + goto out; + +out: +kmem_free (ppi); +return error; +} +int xfs_ioc_swapext (xfs_swapext_t *sxp)@@ -1968,7 +2058,8 @@ xfs_file_ioctl(case XFS_IOC_FSGETXATTRA :return xfs_ioc_fsgetxattra (ip, arg); - +case XFS_IOC_GETPPOINTER : +return xfs_ioc_get_parent_pointer (filp, arg);case XFS_IOC_GETBMAP :case XFS_IOC_GETBMAPA :case XFS_IOC_GETBMAPX :fs/xfs/xfs_ondisk.h +4
-0@@ -135,6 +135,10 @@ xfs_check_ondisk_structs(void) XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 40); XFS_CHECK_STRUCT_SIZE(struct xfs_attrd_log_format, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr, 280); + XFS_CHECK_STRUCT_SIZE(struct xfs_pptr_info, 104); +/* * The v5 superblock format extended several v4 header structures with
fs/xfs/xfs_ondisk.h +0
-0* additional data. While new fields are only accessible on v5 fs/xfs/xfs_parent_utils.c new file +9
-0@@ -0,0 +1,134 @@ +/* +* Copyright (c) 2015 Red Hat, Inc. +* All rights reserved. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
Quoted 8 lines
Quoted 5 lines
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation
+ */Please condense this boilerplate down to a SPDX tag and a copyright statement.
Sure, will do
Quoted 37 lines
Quoted 22 lines
fs/xfs/xfs_parent_utils.c new file +21
-0+#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_bmap_btree.h" +#include "xfs_inode.h" +#include "xfs_error.h" +#include "xfs_trace.h" +#include "xfs_trans.h" +#include "xfs_da_format.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" +#include "xfs_ioctl.h" +#include "xfs_parent.h" +#include "xfs_da_btree.h" + +/* +* Get the parent pointers for a given inode fs/xfs/xfs_parent_utils.c line 47
Quoted 10 lines
fs/xfs/xfs_parent_utils.c new file +10
-0+* +* Returns 0 on success and non zero on error +*/ +int +xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi) + +{ + +struct xfs_attrlist *alist;int
xfs_attr_get_parent_pointer(
structxfs_inode*ip,
structxfs_pptr_info*ppi)
Quoted 2 lines
{
struct xfs_attrlist *alist;will fix
Quoted 18 lines
Quoted 18 lines
fs/xfs/xfs_parent_utils.c new file +8
-0+struct xfs_attrlist_ent *aent; +struct xfs_parent_ptr *xpp; +struct xfs_parent_name_rec *xpnr; +char *namebuf; +unsigned int namebuf_size; +int name_len; +int error =0 ; + unsigned int ioc_flags =
XFS_IOC_ATTR_PARENT;fs/xfs/xfs_parent_utils.c new file +7
-0+unsigned int flags =XFS_ATTR_PARENT ; +int i; +struct xfs_attr_list_context context; + +/* Allocate a buffer to store the attribute names */ + namebuf_size =sizeof (struct xfs_attrlist ) + + (ppi->pi_ptrs_size) * sizeof(struct
Quoted 8 lines
fs/xfs/xfs_parent_utils.c line 65
Quoted 4 lines
xfs_attrlist_ent);fs/xfs/xfs_parent_utils.c new file +3
-0+ namebuf =kvzalloc (namebuf_size,GFP_KERNEL ); +if (!namebuf) +return -ENOMEM ;Do we need the buffer to be zeroed if
xfs_attr_listis just going to set
its contents?
I think i might have initially done this out of habit, but I think it's safe to remove.
Quoted 15 lines
fs/xfs/xfs_parent_utils.c line 69
Quoted 6 lines
fs/xfs/xfs_parent_utils.c new file +3
-0+ +memset (&context,0 ,sizeof (struct xfs_attr_list_context )); + error = xfs_ioc_attr_list_context_init(ip, namebuf,
namebuf_size,fs/xfs/xfs_parent_utils.c new file +1
-0+ ioc_flags, &context);Aha, so the internal implementation has access to
xfs_attr_list_context
before it calls into the attr list code. Ok, in that case,xfs_fs.hdoesn't need theXFS_IOC_ATTR_PARENTflag, and you can set context.attr_filter =XFS_ATTR_PARENThere. Then we don't have to worry
about the existing xattr bulk ioctls returning parent pointers.Oh ok.
I'll see if I can take it out
Oh ok, I'll take a look and see it it can come out.
Quoted 12 lines
fs/xfs/xfs_parent_utils.c line 76
Quoted 8 lines
fs/xfs/xfs_parent_utils.c new file +7
-0+ +/* Copy the cursor provided by caller */ +memcpy (&context.cursor, &ppi->pi_cursor, +sizeof (struct xfs_attrlist_cursor )); + +if (error) + goto out_kfree;Why does the error check come after copying the cursor into the onstack
variable?
Hmm, there might have been a reason at one point, but I think xfs_ioc_attr_list_context_init could actually just be a void return now.
Quoted 5 lines
fs/xfs/xfs_parent_utils.c
Quoted 3 lines
fs/xfs/xfs_parent_utils.c new file +2
-0+ +xfs_ilock (ip,XFS_ILOCK_EXCL );
xfs_ilock_attr_map_shared()?
Ok, will update
Quoted 24 lines
Quoted 24 lines
fs/xfs/xfs_parent_utils.c new file +23
-0+ + error =xfs_attr_list_ilocked (&context); +if (error) + goto out_kfree; + + alist = (struct xfs_attrlist *)namebuf; +for (i =0 ; i < alist->al_count; i++) { +struct xfs_da_args args = { + .geo = ip->i_mount->m_attr_geo, + .whichfork =XFS_ATTR_FORK , + .dp = ip, + .namelen =sizeof (struct xfs_parent_name_rec ), + .attr_filter = flags, + .op_flags =XFS_DA_OP_OKNOENT , + }; + + xpp =xfs_ppinfo_to_pp (ppi, i); +memset (xpp,0 ,sizeof (struct xfs_parent_ptr )); + aent = (struct xfs_attrlist_ent *) + &namebuf[alist->al_offset[i]]; + xpnr = (struct xfs_parent_name_rec *)(aent->a_name); + +if (aent->a_valuelen >XFS_PPTR_MAXNAMELEN ) {
Quoted 8 lines
fs/xfs/xfs_parent_utils.c
Quoted 2 lines
fs/xfs/xfs_parent_utils.c new file +2
-0+ error = -ERANGE ; + goto out_kfree;If a parent pointer has a name longer than MAXNAMELEN then isn't that a
corruption? And in that case,-EFSCORRUPTEDwould be more appropriate
here, right?
I think so, will fix
Quoted 12 lines
fs/xfs/xfs_parent_utils.c line 112
Quoted 11 lines
fs/xfs/xfs_parent_utils.c new file +5
-0+ } + name_len = aent->a_valuelen; + + args.name = (char *)xpnr; + args.hashval = xfs_da_hashname(args.name,args.namelen),
fs/xfs/xfs_parent_utils.c new file +4
-0+ args.value = (unsigned char *)(xpp->xpp_name); + args.valuelen = name_len; + + error =xfs_attr_get_ilocked (&args);
Quoted 7 lines
If error is ENOENT (or ENOATTR or whatever the return value is when the
attr doesn't exist) then shouldn't that be treated as a corruption too?
We still hold the ILOCK from earlier. I don't think OKNOENT is correct
either.
Hmm, I think I likely borrowed this from similar code else where, but if the inode is locked in this case probably any error is grounds for corruption. will update
Quoted 11 lines
fs/xfs/xfs_parent_utils.c line 118
Quoted 7 lines
fs/xfs/xfs_parent_utils.c new file +6
-0+ error = (error == -EEXIST ?0 : error); +if (error) + goto out_kfree; + + xpp->xpp_namelen = name_len; +xfs_init_parent_ptr (xpp, xpnr);Also, should we validate xpnr before copying it out to userspace? If, say, the inode number is bogus, that should generate an EFSCORRUPTED.
I suppose we could validate the inode while we have it here.
Quoted 13 lines
fs/xfs/xfs_parent_utils.c line 126
Quoted 9 lines
fs/xfs/xfs_parent_utils.c new file +8
-0+ } + ppi->pi_ptrs_used = alist->al_count; +if (!alist->al_more) + ppi->pi_flags |=XFS_PPTR_OFLAG_DONE ; + +/* Update the caller with the current cursor position */ +memcpy (&ppi->pi_cursor, &context.cursor, +sizeof (struct xfs_attrlist_cursor ));Glad you remembered to do this; attrmulti forgot to do this for a long
time. :)
:-) I do recall running into it some time ago
Quoted 7 lines
fs/xfs/xfs_parent_utils.c line 130
Quoted 5 lines
fs/xfs/xfs_parent_utils.c new file +4
-0+ +out_kfree: +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +kmem_free (namebuf);kvfree, since you got namebuf from kvzalloc.
Alrighty
Quoted 15 lines
fs/xfs/xfs_parent_utils.h line 2
Quoted 13 lines
fs/xfs/xfs_parent_utils.c new file +4
-0+ +return error; +} +fs/xfs/xfs_parent_utils.h new file +2
-0@@ -0,0 +1,22 @@ +/* +* Copyright (c) 2017 Oracle, Inc. 2022?
Sure, will update date
Quoted 18 lines
Quoted 9 lines
fs/xfs/xfs_parent_utils.h new file +7
-0+* All Rights Reserved. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful,
Quoted 8 lines
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation Inc.
Quoted 2 lines
This also needs to be condensed to a SPDX header and a copyright statement.
Right, will clean that up too
Thanks for the reviews!
Allison
Quoted 11 lines
Quoted 11 lines
+ */
fs/xfs/xfs_parent_utils.h new file +6
-0+#ifndef __XFS_PARENT_UTILS_H__ +#define __XFS_PARENT_UTILS_H__ + +int xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi); +#endif /* __XFS_PARENT_UTILS_H__ */ Signature · 1 line
2.25.1
Re: [PATCH RESEND v2 18/18] xfs: Add parent pointer ioctl
Quoted 67 lines · Alli
Quoted 39 lines · Darrick J. Wong
Quoted 38 lines · Allison Henderson
This patch adds a new file ioctl to retrieve the parent pointer of a
given inodeSigned-off-by Allison Henderson <[email protected]>--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_fs.h | 57 ++++++++++++++++ fs/xfs/libxfs/xfs_parent.c | 10 +++ fs/xfs/libxfs/xfs_parent.h | 2 + fs/xfs/xfs_ioctl.c | 95 +++++++++++++++++++++++++- fs/xfs/xfs_ondisk.h | 4 ++ fs/xfs/xfs_parent_utils.c | 134 +++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 22 ++++++ 8 files changed, 323 insertions(+), 2 deletions(-)fs/xfs/Makefile +1
-0@@ -86,6 +86,7 @@ xfs-y += xfs_aops.o \ xfs_mount.o \ xfs_mru_cache.o \ xfs_pwork.o \ + xfs_parent_utils.o \ xfs_reflink.o \ xfs_stats.o \ xfs_super.o \fs/xfs/libxfs/xfs_fs.h +0
-0@@ -574,6 +574,7 @@ typedef struct xfs_fsop_handlereq {#define XFS_IOC_ATTR_SECURE 0x0008/* use attrs in security namespace */
#defineXFS_IOC_ATTR_CREATE0x0010/* fail if attr
already exists */Quoted 8 lines
Quoted 4 lines
#define
XFS_IOC_ATTR_REPLACE0x0020/* fail if attr
does not exist */
+#defineXFS_IOC_ATTR_PARENT0x0040/* use attrs in parent
namespace */This is the userspace API header, so I wonder -- should we allow
XFS_IOC_ATTRLIST_BY_HANDLEandXFS_IOC_ATTRMULTI_BY_HANDLEto access parent pointers?Well, the ioc is how the test cases get the pptrs back out in order to verify parent pointers are working. So we need to keep at least that, but then I think it makes worrying about other forms of access feel sort of silly since we're not really hiding anything. They would have to pass in the parent filter flag which wasnt allowable until now, so it's not like having pptrs appear in the list when asked for is inappropriate.
Quoted 4 lines
I think it's *definitely* incorrect to let
ATTR_OP_REMOVEorATTR_OP_SET
(attrmulti subcommands) to mess with parent pointers.Ok, I can see if I can add some sanity checking there.
Quoted 5 lines
I don't think attrlist or
ATTR_OP_GETshould be touching them either, particularly since you're defining a new ioctl to extract *only* the parent pointers.
Quoted 8 lines
Quoted 6 lines
If there wasn't
XFS_IOC_GETPPOINTERthen perhaps it would be ok to allow
reads via ATTRLIST/ATTRMULTI. But even then, I don't think we want things like xfsdump to think that it has to preserve those attributes since xfsrestore will reconstruct the directory tree (and hence the pptrs) for us.Hrmm, not sure I follow this part, the point of pptrs are to reconstruct the tree, so wouldnt we want them preserved?
Parent pointers are backwards links through the directory tree. xfsdump already records the forward links in the dump file. xfsrestore uses those forward links to rebuild the directory tree, which recreates the parent pointers automatically. Hence we don't need ATTRMULTI to reveal (or recreate) the parent pointer xattrs; the kernel does that when we create the directory tree.
The second reason I can think of why we don't want to expose the parent pointers through the xattr APIs is that we don't want to reveal ondisk metadata directly to users -- some day we might want to change wthat's stored on disk, or store them in a totally separate structure, or whatever.
If we force the interface to be the GETPARENTS ioctl, then we've decoupled the front and backends. I conclude that the /only/ userspace API that should ever touch parent pointers is XFS_IOC_GETPARENTS.
I'll forward this on to the v3 thread too.
--D
Quoted 678 lines
Quoted 37 lines
Quoted 31 lines
typedef struct
xfs_attrlist_cursor{fs/xfs/libxfs/xfs_fs.h +15
-0__u32 opaque[4 ];@@ -752,6 +753,61 @@ struct xfs_scrub_metadata { XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED)#define XFS_SCRUB_FLAGS_ALL (XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT) +#define XFS_PPTR_MAXNAMELEN 256 + +/* return parents of the handle, not the open fd */ +#define XFS_PPTR_IFLAG_HANDLE (1U << 0) + +/* target was the root directory */ +#define XFS_PPTR_OFLAG_ROOT (1U << 1) + +/* Cursor is done iterating pptrs */ +#define XFS_PPTR_OFLAG_DONE (1U << 2) + +/* Get an inode parent pointer through ioctl */ +struct xfs_parent_ptr { +__u64 xpp_ino;/* Inode */ +__u32 xpp_gen;/* Inode generation */
fs/xfs/libxfs/xfs_fs.h +1
-0+__u32 xpp_diroffset;/* Directory offset */
fs/xfs/libxfs/xfs_fs.h +1
-0+__u32 xpp_namelen;/* File name length */
fs/xfs/libxfs/xfs_fs.h +2
-0+__u32 xpp_pad; +__u8 xpp_name[XFS_PPTR_MAXNAMELEN ];/* File Quoted 1 line
name */
Since
xpp_nameis a fixed-length array that is long enough to ensure that there's a null at the end of the name, we don't needxpp_namelen.Quoted 3 lines
I wonder if
xpp_namelenandxpp_padshould simply turn into a u64 field
that's defined zero for future expansion?Sure, I'll see if I can remove it and add a reserved field
Quoted 12 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 783
Quoted 10 lines
fs/xfs/libxfs/xfs_fs.h +9
-0+}; + +/* Iterate through an inodes parent pointers */ +struct xfs_pptr_info { +struct xfs_handle pi_handle; +struct xfs_attrlist_cursor pi_cursor; +__u32 pi_flags; +__u32 pi_reserved; +__u32 pi_ptrs_size;Is this the number of elements in pi_parents[]?
Yes, it's the number parent pointers in the array
Quoted 19 lines
Quoted 19 lines
fs/xfs/libxfs/xfs_fs.h +18
-0+__u32 pi_ptrs_used; +__u64 pi_reserved2[6 ]; + +/* +* An array of struct xfs_parent_ptr follows the header +* information. Use XFS_PPINFO_TO_PP() to access the +* parent pointer array entries. +*/ +struct xfs_parent_ptr pi_parents[]; +}; + +static inline size_t +xfs_pptr_info_sizeof (int nr_ptrs) +{ +return sizeof (struct xfs_pptr_info ) + + (nr_ptrs *sizeof (struct xfs_parent_ptr )); +} +Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 807
Quoted 6 lines
fs/xfs/libxfs/xfs_fs.h +6
-0+static inline struct xfs_parent_ptr * +xfs_ppinfo_to_pp ( +struct xfs_pptr_info *info, +int idx) +{ +Nit: extra space.
Will fix
Quoted 20 lines
Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h +3
-0+return &info->pi_parents[idx]; +} +/* * ioctl limits */ fs/xfs/libxfs/xfs_fs.h struct xfs_scrub_metadata { · line 859
Quoted 7 lines
fs/xfs/libxfs/xfs_fs.h +1
-0@@ -797,6 +853,7 @@ struct xfs_scrub_metadata {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +#define XFS_IOC_GETPPOINTER _IOR ('X', 62, struct xfs_parent_ptr)I wonder if this name should more strongly emphasize that it's for reading
the parents of a file?Quoted 1 line
#define
XFS_IOC_GETPARENTS_IOWR(...)Sure, that sounds fine i think
Quoted 5 lines
Also, the ioctl reads and writes its parameter, so this is
_IOWR, not_IOR.BTW, is there a sample manpage somewhere?
The userspace branch adds some new flags to xfsprogs and some usage help to explain how to use them. See the last patch in the branch:
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrsv2But it's just for printing the parent pointers out, it doesn't have a man page for how to write your own ioctl. I suppose we could add it though.
Quoted 13 lines
Quoted 13 lines
/*
* ioctl commands that replace IRIX syssgi()'sfs/xfs/libxfs/xfs_parent.c +0
-0@@ -26,6 +26,16 @@#include "xfs_xattr.h" #include "xfs_parent.h" Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c line 32
Quoted 4 lines
fs/xfs/libxfs/xfs_parent.c +4
-0+/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */ +void +xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec)The second parameter ought to be const struct
xfs_parent_name_rec*rec
to make it unambiguous to readers which is the source and which is the
destination argument.Ok, will update
Quoted 46 lines
Quoted 46 lines
fs/xfs/libxfs/xfs_parent.c +6
-0+{ + xpp->xpp_ino =be64_to_cpu (rec->p_ino); + xpp->xpp_gen =be32_to_cpu (rec->p_gen); + xpp->xpp_diroffset =be32_to_cpu (rec->p_diroffset); +} +/* * Parent pointer attribute handling. * fs/xfs/libxfs/xfs_parent.h +2
-0@@ -23,6 +23,8 @@ void xfs_init_parent_name_rec(structxfs_parent_name_rec *rec, uint32_t p_diroffset);void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec,struct xfs_parent_name_rec *rec); +void xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +struct xfs_parent_name_rec *rec); int xfs_parent_init(xfs_mount_t *mp, xfs_inode_t *ip, struct xfs_name *target_name,fs/xfs/libxfs/xfs_parent.h +0
-0struct xfs_parent_defer **parentp);fs/xfs/xfs_ioctl.c +3
-0@@ -37,6 +37,7 @@#include "xfs_health.h" #include "xfs_reflink.h" #include "xfs_ioctl.h" +#include "xfs_parent_utils.h" #include "xfs_xattr.h" #include <linux/mount.h> @@ -355,6 +356,8 @@ xfs_attr_filter(return XFS_ATTR_ROOT ;if (ioc_flags &XFS_IOC_ATTR_SECURE )return XFS_ATTR_SECURE ; +if (ioc_flags &XFS_IOC_ATTR_PARENT ) +return XFS_ATTR_PARENT ;return 0 ; }Quoted 13 lines
fs/xfs/xfs_ioctl.c xfs_ioc_attr_list( · line 430
Quoted 8 lines
fs/xfs/xfs_ioctl.c +2
-1@@ -422,7 +425,8 @@ xfs_ioc_attr_list(/* * Reject flags, only allow namespaces. */ -if (flags & ~(XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE )) +if (flags & ~(XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE | +XFS_IOC_ATTR_PARENT ))return -EINVAL ;I think
xfs_ioc_attrmulti_oneneeds filtering forXFS_IOC_ATTR_PARENT,
if we're still going to allow attrlist/attrmulti to return parent pointers.Ok, will update that one as well then
Quoted 33 lines
Quoted 33 lines
fs/xfs/xfs_ioctl.c +2
-0if (flags == (XFS_IOC_ATTR_ROOT |XFS_IOC_ATTR_SECURE ))return -EINVAL ;@@ -1679,6 +1683,92 @@ xfs_ioc_scrub_metadata(return 0 ; } +/* +* IOCTL routine to get the parent pointers of an inode and return it to user
+ * space. Caller must pass a buffer space containing a structxfs_pptr_info,
+ * followed by a region large enough to contain an array of struct
+ *xfs_parent_ptrof a size specified inpi_ptrs_size. If the inode contains
+ * more parent pointers than can fit in the buffer space, caller may re-call
+ * the function using the returnedpi_cursorto resume iteration. The
+ * number ofxfs_parent_ptrreturned will be stored inpi_ptrs_used.
+ *
+ * Returns 0 on success or non-zero on failure
+ */fs/xfs/xfs_ioctl.c +9
-0+STATIC int +xfs_ioc_get_parent_pointer ( +struct file *filp, +void __user *arg) +{ +struct xfs_pptr_info *ppi =NULL ; +int error =0 ; +struct xfs_inode *ip =XFS_I (file_inode (filp)); +struct xfs_mount *mp = ip->i_mount;Quoted 9 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1711
Quoted 6 lines
fs/xfs/xfs_ioctl.c +6
-0+ +if (!capable (CAP_SYS_ADMIN )) +return -EPERM ; + +/* Allocate an xfs_pptr_info to put the user data */ + ppi =kmem_alloc (sizeof (struct xfs_pptr_info ),0 );New code should call kmalloc instead of the old
kmem_allocwrapper.Ok, will update
Quoted 9 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1716
Quoted 5 lines
fs/xfs/xfs_ioctl.c +5
-0+if (!ppi) +return -ENOMEM ; + +/* Copy the data from the user */ + error =copy_from_user (ppi, arg,sizeof (struct xfs_pptr_info ));Note:
copy_from_userreturns the number of bytes *not* copied. If you
receive a nonzero return value, error usually gets set to EFAULT.ooooh. ok, will fix that then.
Quoted 26 lines
Quoted 12 lines
fs/xfs/xfs_ioctl.c +5
-0+if (error) + goto out; + +/* Check size of buffer requested by user */ + if (xfs_pptr_info_sizeof(ppi->pi_ptrs_size) >
XFS_XATTR_LIST_MAX) {fs/xfs/xfs_ioctl.c +5
-0+ error = -ENOMEM ; + goto out; + } + + if (ppi->pi_flags != 0 && ppi->pi_flags !=Quoted 1 line
XFS_PPTR_IFLAG_HANDLE) {if (ppi->pi_flags & ~XFS_PPTR_IFLAG_HANDLE) ?(If we really want to be pedantic, this really ought to be:
#define
XFS_PPTR_IFLAG_ALL(XFS_PPTR_IFLAG_HANDLE)if (ppi->pi_flags & ~XFS_PPTR_IFLAG_ALL) return -EINVAL;Or you could be more flexible, since the kernel could just set the OFLAGs appropriately and not care about their value on input:
Quoted 7 lines
#define
XFS_PPTR_FLAG_ALL(XFS_PPTR_IFLAG_HANDLE|XFS_PPTR_OFLAG...)if (ppi->pi_flags & ~XFS_PPTR_FLAG_ALL) return -EINVAL; ppi->pi_flags &= ~(XFS_PPTR_OFLAG_ROOT | XFS_PPTR_OFLAG_DONE);Oh, I see, sure that makes sense.
Quoted 16 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1738
Quoted 13 lines
fs/xfs/xfs_ioctl.c +12
-0+ error = -EINVAL ; + goto out; + } + +/* +* Now that we know how big the trailing buffer is, expand +* our kernel xfs_pptr_info to be the same size +*/ + ppi =krealloc (ppi,xfs_pptr_info_sizeof (ppi->pi_ptrs_size), +GFP_NOFS | __GFP_NOFAIL); +if (!ppi) +return -ENOMEM ;Why NOFS and NOFAIL? We don't have any writeback resources locked (transactions and ILOCKs) so we can hit ourselves up for memory.
Ok, will update
Quoted 5 lines
fs/xfs/xfs_ioctl.c
Quoted 3 lines
fs/xfs/xfs_ioctl.c +2
-0+ +if (ppi->pi_flags ==XFS_PPTR_IFLAG_HANDLE ) {if (ppi->pi_flags & XFS_PPTR_IFLAG_HANDLE) {ok, will fix
Quoted 20 lines
Quoted 20 lines
fs/xfs/xfs_ioctl.c +1
-0+ error = xfs_iget(mp, NULL, ppi-Quoted 1 line
pi_handle.ha_fid.fid_ino,
fs/xfs/xfs_ioctl.c +5
-0+ 0, 0, &ip); +if (error) + goto out; + + if (VFS_I(ip)->i_generation != ppi-Quoted 1 line
pi_handle.ha_fid.fid_gen) {
fs/xfs/xfs_ioctl.c +11
-0+ error = -EINVAL ; + goto out; + } + } + +if (ip->i_ino == mp->m_sb.sb_rootino) + ppi->pi_flags |=XFS_PPTR_OFLAG_ROOT ; + +/* Get the parent pointers */ + error =xfs_attr_get_parent_pointer (ip, ppi); +Quoted 9 lines
fs/xfs/xfs_ioctl.c xfs_ioc_scrub_metadata( · line 1763
Quoted 6 lines
fs/xfs/xfs_ioctl.c +6
-0+if (error) + goto out; + +/* Copy the parent pointers back to the user */ + error =copy_to_user (arg, ppi, +xfs_pptr_info_sizeof (ppi->pi_ptrs_size));Same note as the one I made for
copy_from_user.Will update
Quoted 57 lines
Quoted 57 lines
fs/xfs/xfs_ioctl.c +10
-1+if (error) + goto out; + +out: +kmem_free (ppi); +return error; +} +int xfs_ioc_swapext (xfs_swapext_t *sxp)@@ -1968,7 +2058,8 @@ xfs_file_ioctl(case XFS_IOC_FSGETXATTRA :return xfs_ioc_fsgetxattra (ip, arg); - +case XFS_IOC_GETPPOINTER : +return xfs_ioc_get_parent_pointer (filp, arg);case XFS_IOC_GETBMAP :case XFS_IOC_GETBMAPA :case XFS_IOC_GETBMAPX :fs/xfs/xfs_ondisk.h +4
-0@@ -135,6 +135,10 @@ xfs_check_ondisk_structs(void) XFS_CHECK_STRUCT_SIZE(struct xfs_attri_log_format, 40); XFS_CHECK_STRUCT_SIZE(struct xfs_attrd_log_format, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr, 280); + XFS_CHECK_STRUCT_SIZE(struct xfs_pptr_info, 104); +/* * The v5 superblock format extended several v4 header structures with
fs/xfs/xfs_ondisk.h +0
-0* additional data. While new fields are only accessible on v5 fs/xfs/xfs_parent_utils.c new file +9
-0@@ -0,0 +1,134 @@ +/* +* Copyright (c) 2015 Red Hat, Inc. +* All rights reserved. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *Quoted 8 lines
Quoted 5 lines
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation
+ */Please condense this boilerplate down to a SPDX tag and a copyright statement.
Sure, will do
Quoted 37 lines
Quoted 22 lines
fs/xfs/xfs_parent_utils.c new file +21
-0+#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_bmap_btree.h" +#include "xfs_inode.h" +#include "xfs_error.h" +#include "xfs_trace.h" +#include "xfs_trans.h" +#include "xfs_da_format.h" +#include "xfs_da_btree.h" +#include "xfs_attr.h" +#include "xfs_ioctl.h" +#include "xfs_parent.h" +#include "xfs_da_btree.h" + +/* +* Get the parent pointers for a given inode fs/xfs/xfs_parent_utils.c line 47
Quoted 10 lines
fs/xfs/xfs_parent_utils.c new file +10
-0+* +* Returns 0 on success and non zero on error +*/ +int +xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi) + +{ + +struct xfs_attrlist *alist;int
xfs_attr_get_parent_pointer(
structxfs_inode*ip,
structxfs_pptr_info*ppi)Quoted 2 lines
{ struct xfs_attrlist *alist;will fix
Quoted 18 lines
Quoted 18 lines
fs/xfs/xfs_parent_utils.c new file +8
-0+struct xfs_attrlist_ent *aent; +struct xfs_parent_ptr *xpp; +struct xfs_parent_name_rec *xpnr; +char *namebuf; +unsigned int namebuf_size; +int name_len; +int error =0 ; + unsigned int ioc_flags =
XFS_IOC_ATTR_PARENT;fs/xfs/xfs_parent_utils.c new file +7
-0+unsigned int flags =XFS_ATTR_PARENT ; +int i; +struct xfs_attr_list_context context; + +/* Allocate a buffer to store the attribute names */ + namebuf_size =sizeof (struct xfs_attrlist ) + + (ppi->pi_ptrs_size) * sizeof(structQuoted 8 lines
fs/xfs/xfs_parent_utils.c line 65
Quoted 4 lines
xfs_attrlist_ent);fs/xfs/xfs_parent_utils.c new file +3
-0+ namebuf =kvzalloc (namebuf_size,GFP_KERNEL ); +if (!namebuf) +return -ENOMEM ;Do we need the buffer to be zeroed if
xfs_attr_listis just going to set
its contents?I think i might have initially done this out of habit, but I think it's safe to remove.
Quoted 15 lines
fs/xfs/xfs_parent_utils.c line 69
Quoted 6 lines
fs/xfs/xfs_parent_utils.c new file +3
-0+ +memset (&context,0 ,sizeof (struct xfs_attr_list_context )); + error = xfs_ioc_attr_list_context_init(ip, namebuf,
namebuf_size,fs/xfs/xfs_parent_utils.c new file +1
-0+ ioc_flags, &context);Aha, so the internal implementation has access to
xfs_attr_list_context
before it calls into the attr list code. Ok, in that case,xfs_fs.hdoesn't need theXFS_IOC_ATTR_PARENTflag, and you can set context.attr_filter =XFS_ATTR_PARENThere. Then we don't have to worry
about the existing xattr bulk ioctls returning parent pointers.Oh ok.
I'll see if I can take it outOh ok, I'll take a look and see it it can come out.
Quoted 12 lines
fs/xfs/xfs_parent_utils.c line 76
Quoted 8 lines
fs/xfs/xfs_parent_utils.c new file +7
-0+ +/* Copy the cursor provided by caller */ +memcpy (&context.cursor, &ppi->pi_cursor, +sizeof (struct xfs_attrlist_cursor )); + +if (error) + goto out_kfree;Why does the error check come after copying the cursor into the onstack
variable?Hmm, there might have been a reason at one point, but I think
xfs_ioc_attr_list_context_initcould actually just be a void return now.Quoted 5 lines
fs/xfs/xfs_parent_utils.c
Quoted 3 lines
fs/xfs/xfs_parent_utils.c new file +2
-0+ +xfs_ilock (ip,XFS_ILOCK_EXCL );
xfs_ilock_attr_map_shared()?Ok, will update
Quoted 24 lines
Quoted 24 lines
fs/xfs/xfs_parent_utils.c new file +23
-0+ + error =xfs_attr_list_ilocked (&context); +if (error) + goto out_kfree; + + alist = (struct xfs_attrlist *)namebuf; +for (i =0 ; i < alist->al_count; i++) { +struct xfs_da_args args = { + .geo = ip->i_mount->m_attr_geo, + .whichfork =XFS_ATTR_FORK , + .dp = ip, + .namelen =sizeof (struct xfs_parent_name_rec ), + .attr_filter = flags, + .op_flags =XFS_DA_OP_OKNOENT , + }; + + xpp =xfs_ppinfo_to_pp (ppi, i); +memset (xpp,0 ,sizeof (struct xfs_parent_ptr )); + aent = (struct xfs_attrlist_ent *) + &namebuf[alist->al_offset[i]]; + xpnr = (struct xfs_parent_name_rec *)(aent->a_name); + +if (aent->a_valuelen >XFS_PPTR_MAXNAMELEN ) {Quoted 8 lines
fs/xfs/xfs_parent_utils.c
Quoted 2 lines
fs/xfs/xfs_parent_utils.c new file +2
-0+ error = -ERANGE ; + goto out_kfree;If a parent pointer has a name longer than MAXNAMELEN then isn't that a
corruption? And in that case,-EFSCORRUPTEDwould be more appropriate
here, right?I think so, will fix
Quoted 12 lines
fs/xfs/xfs_parent_utils.c line 112
Quoted 11 lines
fs/xfs/xfs_parent_utils.c new file +5
-0+ } + name_len = aent->a_valuelen; + + args.name = (char *)xpnr; + args.hashval = xfs_da_hashname(args.name,args.namelen),
fs/xfs/xfs_parent_utils.c new file +4
-0+ args.value = (unsigned char *)(xpp->xpp_name); + args.valuelen = name_len; + + error =xfs_attr_get_ilocked (&args);Quoted 7 lines
If error is ENOENT (or ENOATTR or whatever the return value is when the
attr doesn't exist) then shouldn't that be treated as a corruption too?
We still hold the ILOCK from earlier. I don't think OKNOENT is correct
either.Hmm, I think I likely borrowed this from similar code else where, but if the inode is locked in this case probably any error is grounds for corruption. will update
Quoted 11 lines
fs/xfs/xfs_parent_utils.c line 118
Quoted 7 lines
fs/xfs/xfs_parent_utils.c new file +6
-0+ error = (error == -EEXIST ?0 : error); +if (error) + goto out_kfree; + + xpp->xpp_namelen = name_len; +xfs_init_parent_ptr (xpp, xpnr);Also, should we validate xpnr before copying it out to userspace? If, say, the inode number is bogus, that should generate an EFSCORRUPTED.
I suppose we could validate the inode while we have it here.
Quoted 13 lines
fs/xfs/xfs_parent_utils.c line 126
Quoted 9 lines
fs/xfs/xfs_parent_utils.c new file +8
-0+ } + ppi->pi_ptrs_used = alist->al_count; +if (!alist->al_more) + ppi->pi_flags |=XFS_PPTR_OFLAG_DONE ; + +/* Update the caller with the current cursor position */ +memcpy (&ppi->pi_cursor, &context.cursor, +sizeof (struct xfs_attrlist_cursor ));Glad you remembered to do this; attrmulti forgot to do this for a long
time. :):-) I do recall running into it some time ago
Quoted 7 lines
fs/xfs/xfs_parent_utils.c line 130
Quoted 5 lines
fs/xfs/xfs_parent_utils.c new file +4
-0+ +out_kfree: +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +kmem_free (namebuf);kvfree, since you got namebuf from kvzalloc.
Alrighty
Quoted 15 lines
fs/xfs/xfs_parent_utils.h line 2
Quoted 13 lines
fs/xfs/xfs_parent_utils.c new file +4
-0+ +return error; +} +fs/xfs/xfs_parent_utils.h new file +2
-0@@ -0,0 +1,22 @@ +/* +* Copyright (c) 2017 Oracle, Inc. 2022?
Sure, will update date
Quoted 18 lines
Quoted 9 lines
fs/xfs/xfs_parent_utils.h new file +7
-0+* All Rights Reserved. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful,
Quoted 8 lines
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation Inc.Quoted 2 lines
This also needs to be condensed to a SPDX header and a copyright statement.
Right, will clean that up too
Thanks for the reviews!
AllisonQuoted 11 lines
Quoted 11 lines
+ */
fs/xfs/xfs_parent_utils.h new file +6
-0+#ifndef __XFS_PARENT_UTILS_H__ +#define __XFS_PARENT_UTILS_H__ + +int xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi); +#endif /* __XFS_PARENT_UTILS_H__ */ Signature · 1 line
2.25.1
[RFC PATCH 19/18] xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
In reply to Allison HendersonFrom: Darrick J. Wong <[email protected]>
Dave and I were discussing some recent test regressions as a result of me turning on nrext64=1 on realtime filesystems, when we noticed that the minimum log size of a 32M filesystem jumped from 954 blocks to 4287 blocks.
Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size contains the maximum estimated amount of space needed for a local format xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES, which requires units of blocks. This has resulted in an overestimation of the minimum log size over the years.
We should nominally correct this, but there's a backwards compatibility problem -- if we enable it now, the minimum log size will decrease. If a corrected mkfs formats a filesystem with this new smaller log size, a user will encounter mount failures on an uncorrected kernel due to the larger minimum log size computations there.
However, the large extent counters feature is still EXPERIMENTAL, so we can gate the correction on that feature (or any features that get added after that) being enabled. Any filesystem with nrext64 or any of the as-yet-undefined feature bits turned on will be rejected by old uncorrected kernels, so this should be safe even in the upgrade case.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 43 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +43 -0
@@ -16,6 +16,39 @@#include "xfs_bmap_btree.h" #include "xfs_trace.h" +/* +* Decide if the filesystem has the parent pointer feature or any feature +* added after that. +*/ +static inline bool +xfs_has_parent_or_newer_feature ( +struct xfs_mount *mp) +{ +if (!xfs_sb_is_v5 (&mp->m_sb)) +return false; + +if (xfs_sb_has_compat_feature (&mp->m_sb, ~0 )) +return true; + +if (xfs_sb_has_ro_compat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_RO_COMPAT_FINOBT | +XFS_SB_FEAT_RO_COMPAT_RMAPBT | +XFS_SB_FEAT_RO_COMPAT_REFLINK | +XFS_SB_FEAT_RO_COMPAT_INOBTCNT ))) +return true; + +if (xfs_sb_has_incompat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_INCOMPAT_FTYPE | +XFS_SB_FEAT_INCOMPAT_SPINODES | +XFS_SB_FEAT_INCOMPAT_META_UUID | +XFS_SB_FEAT_INCOMPAT_BIGTIME | +XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR | +XFS_SB_FEAT_INCOMPAT_NREXT64 ))) +return true; + +return false; +} +/* * Calculate the maximum length in bytes that would be required for a local * attribute value as large attributes out of line are not logged.
@@ -31,6 +64,16 @@ xfs_log_calc_max_attrsetm_res(MAXNAMELEN -1 ; nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); nblks +=XFS_B_TO_FSB (mp, size); + +/* +* Starting with the parent pointer feature, every new fs feature +* corrects a unit conversion error in the xattr transaction +* reservation code that resulted in oversized minimum log size +* computations. +*/ +if (xfs_has_parent_or_newer_feature (mp)) + size =XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); return M_RES(mp)->tr_attrsetm.tr_logres +
[RFC PATCH 20/18] xfs: drop compatibility minimum log size computations for reflink
In reply to Allison HendersonFrom: Darrick J. Wong <[email protected]>
Having established that we can reduce the minimum log size computation for filesystems with parent pointers or any newer feature, we should also drop the compat minlogsize code that we added when we reduced the transaction reservation size for rmap and reflink.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +10 -0
@@ -91,6 +91,16 @@ xfs_log_calc_trans_resv_for_minlogblocks( {unsigned int rmap_maxlevels = mp->m_rmap_maxlevels; +/* +* Starting with the parent pointer feature, every new fs feature +* drops the oversized minimum log size computation introduced by the +* original reflink code. +*/ +if (xfs_has_parent_or_newer_feature (mp)) { +xfs_trans_resv_calc (mp, resv); +return ; + } +/* * In the early days of rmap+reflink, we always set the rmap maxlevels * to 9 even if the AG was small enough that it would never grow to