xfs: Parent Pointers
PATCH v11 1/1 xfs: Parent Pointers
Hi everyone,
This submission contains all of the changes to the parent pointers patchset that I've been working since last month's deluge. The kernel and xfsprogs patchsets are based on Allison's v10 tag from some time ago. To recap Allison's cover letter:
"The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point."
v11 rebases everything against 6.3-rc5, and weaves all the changes that I had made against v10 into Allison's original series. The new xattr NVLOOKUP mode that I introduced for v10 is critical for handling parent pointer attr name collisions with grace, so that has been retained from v10. With that in place, I've replaced the diroffset in the ondisk parent pointer with the dirent hash of the name.
Parent pointers now look like this:
(parent_ino, parent_gen, namehash) -> (name[])
I experimented with replacing the dahash with crc32c for this patchset but left it out, having concluded that checksum operation has higher overhead (thanks, crypto api!), would require someone to figure out crc spoofing sufficiently well to keep metadump name obfuscation working, and doesn't seem to improve collision resistance sufficiently to be worth the added engineering cost.
As of this submission, I think this feature is ready to go once we've merged the online repair code and rebased the online repair code to actually commit the repaired directories.
If you want to pull the whole thing, use these links:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
--D
Quoted 32 lines · Darrick J. Wong
Hi everyone,
This submission contains all of the changes to the parent pointers patchset that I've been working since last month's deluge. The kernel and xfsprogs patchsets are based on Allison's v10 tag from some time ago. To recap Allison's cover letter:
"The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point."
v11 rebases everything against 6.3-rc5, and weaves all the changes that I had made against v10 into Allison's original series. The new xattr NVLOOKUP mode that I introduced for v10 is critical for handling parent pointer attr name collisions with grace, so that has been retained from v10. With that in place, I've replaced the diroffset in the ondisk parent pointer with the dirent hash of the name.
Parent pointers now look like this:
(parent_ino, parent_gen, namehash) -> (name[])I experimented with replacing the dahash with crc32c for this patchset but left it out, having concluded that checksum operation has higher overhead (thanks, crypto api!), would require someone to figure out crc spoofing sufficiently well to keep metadump name obfuscation working, and doesn't seem to improve collision resistance sufficiently to be worth the added engineering cost.
Quoted 7 lines
As of this submission, I think this feature is ready to go once we've merged the online repair code and rebased the online repair code to actually commit the repaired directories.
If you want to pull the whole thing, use these links:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary
<sigh> Apparently vim undid my edit here; the correct URL is:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs
--D
v11 cover xfs: name-value xattr lookups
In reply to Darrick J. WongHi all,
Directory parent pointers are stored as namespaced extended attributes of a file. Because parent pointers can consume up to 267 bytes of space and xattr names are 255 bytes at most, we cannot use the usual attr name lookup functions to find a parent pointer. This is solvable by introducing a new lookup mode that checks both the name and the value of the xattr.
Therefore, introduce this new lookup mode. Because all parent pointer updates are logged, we must extend the xattr logging code to capture the VLOOKUP variants, and restore them when recovering logged operations. These new log formats are protected by the sb_incompat PARENT flag, so they do not need a separate log_incompat feature flag.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-attr-nvlookups
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-attr-nvlookups
--- fs/xfs/libxfs/xfs_attr.c | 57 +++++-- fs/xfs/libxfs/xfs_attr.h | 9 + fs/xfs/libxfs/xfs_attr_leaf.c | 45 ++++- fs/xfs/libxfs/xfs_da_btree.h | 10 + fs/xfs/libxfs/xfs_log_format.h | 30 +++ fs/xfs/xfs_attr_item.c | 347 ++++++++++++++++++++++++++++++++++------ fs/xfs/xfs_attr_item.h | 2 fs/xfs/xfs_xattr.c | 5 + 8 files changed, 425 insertions(+), 80 deletions(-)
PATCH v11 1/12 xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2
From: Darrick J. Wong <[email protected]>
Check that the number of recovered log iovecs is what is expected for the xattri opcode is expecting.
--- fs/xfs/xfs_attr_item.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
fs/xfs/xfs_attr_item.c +27 -0
@@ -710,6 +710,7 @@ xlog_recover_attri_commit_pass2(const void *attr_value =NULL ;const void *attr_name;size_t len; +unsigned int op; attri_formatp = item->ri_buf[0 ].i_addr; attr_name = item->ri_buf[1 ].i_addr;
@@ -728,6 +729,32 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } +/* Check the number of log iovecs makes sense for the op code. */ + op = attri_formatp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +switch (op) { +case XFS_ATTRI_OP_FLAGS_SET : +case XFS_ATTRI_OP_FLAGS_REPLACE : +/* Log item, attr name, attr value */ +if (item->ri_total !=3 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +break ; +case XFS_ATTRI_OP_FLAGS_REMOVE : +/* Log item, attr name */ +if (item->ri_total !=2 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +break ; +default : +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +/* Validate the attr name */ if (item->ri_buf[1 ].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_name_len)) {
PATCH v11 2/12 xfs: make xfs_attr_set require XFS_DA_OP_REMOVE
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Quite a few patches from now, we're going to change the parent pointer xattr format to encode as much of the dirent name in the xattr name as fits, and spill the rest of it to the xattr value. To make this work correctly, we'll be adding the ability to look up xattrs based on name /and/ value.
Internally, the xattr data structure supports attributes with a zero length value, which is how we're going to store parent pointers for short dirent names. The parent pointer repair code uses xfs_attr_set to add missing and remove dangling parent pointers, so that interface must be capable of setting an xattr with args->value == NULL.
The userspace API doesn't support this, so xfs_attr_set currently treats a NULL args->value as a request to remove an attr. However, that's a quirk of the existing callers and the interface. Make the callers of xfs_attr_set to declare explicitly that they want to remove an xattr.
--- fs/xfs/libxfs/xfs_attr.c | 9 +++++---- fs/xfs/xfs_xattr.c | 5 +++++ 2 files changed, 10 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +5 -4
@@ -967,6 +967,7 @@ xfs_attr_set(struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres;bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool is_remove = args->op_flags &XFS_DA_OP_REMOVE ;int error, local;int rmt_blks =0 ;unsigned int total;
@@ -991,7 +992,7 @@ xfs_attr_set( args->op_flags =XFS_DA_OP_OKNOENT | (args->op_flags &XFS_DA_OP_LOGGED ); -if (args->value) { +if (!is_remove) {XFS_STATS_INC (mp, xs_attr_set); args->total =xfs_attr_calc_size (args, &local);
@@ -1025,7 +1026,7 @@ xfs_attr_set(if (error)return error; -if (args->value ||xfs_inode_hasattr (dp)) { +if (!is_remove ||xfs_inode_hasattr (dp)) { error =xfs_iext_count_may_overflow (dp,XFS_ATTR_FORK ,XFS_IEXT_ATTR_MANIP_CNT (rmt_blks));if (error == -EFBIG )
@@ -1039,7 +1040,7 @@ xfs_attr_set(switch (error) {case -EEXIST :/* if no value, we are performing a remove operation */ -if (!args->value) { +if (is_remove) { error =xfs_attr_defer_remove (args);break ; }
@@ -1051,7 +1052,7 @@ xfs_attr_set(break ;case -ENOATTR :/* Can't remove what isn't there. */ -if (!args->value) +if (is_remove) goto out_trans_cancel;/* Pure replace fails if no existing attr to replace. */
fs/xfs/xfs_xattr.c +5 -0
@@ -103,6 +103,11 @@ xfs_attr_change( use_logging = true; } +if (args->value) + args->op_flags &= ~XFS_DA_OP_REMOVE ; +else + args->op_flags |=XFS_DA_OP_REMOVE ; + error =xfs_attr_set (args);if (use_logging)
PATCH v11 3/12 xfs: allow xattr matching on name and value for local/sf attrs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a new NVLOOKUP flag to signal that the caller wants to look up an extended attribute by name and value. This only works with shortform and local attributes. Only parent pointers need this functionality and parent pointers cannot be remote xattrs, so this limitation is ok for now.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 45 ++++++++++++++++++++++++++++++++++------- fs/xfs/libxfs/xfs_da_btree.h | 4 +++- 2 files changed, 40 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +37 -8
@@ -473,10 +473,12 @@ xfs_attr3_leaf_read(*/ static bool xfs_attr_match ( -struct xfs_da_args *args, -uint8_t namelen, -unsigned char *name, -int flags) +const struct xfs_da_args *args, +uint8_t namelen, +const unsigned char *name, +unsigned int valuelen, +const void *value, +int flags) {if (args->namelen != namelen)
@@ -484,6 +486,23 @@ xfs_attr_match(if (memcmp (args->name, name, namelen) !=0 )return false; +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) { +if (args->valuelen != valuelen) +return false; +if (args->valuelen && !value) { +/* not implemented for remote values */ +ASSERT (0 ); +return false; + } +if (valuelen && !args->value) { +/* caller gave us valuelen > 0 but no value?? */ +ASSERT (0 ); +return false; + } +if (valuelen >0 &&memcmp (args->value, value, valuelen) !=0 ) +return false; + } +/* Recovery ignores the INCOMPLETE flag. */ if ((args->op_flags &XFS_DA_OP_RECOVERY ) && args->attr_filter == (flags &XFS_ATTR_NSP_ONDISK_MASK ))
@@ -502,6 +521,10 @@ xfs_attr_copy_value( unsigned char *value, int valuelen) { +/* vlookups already supplied the attr value; don't copy anything */ +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) +return 0 ; +/* * No copy if all we have to do is get the length */
@@ -726,6 +749,7 @@ xfs_attr_sf_findname( base += size, i++) { size =xfs_attr_sf_entsize (sfe);if (!xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))continue ;break ;
@@ -894,6 +918,7 @@ xfs_attr_shortform_lookup(xfs_da_args_t *args)for (i =0 ; i < sf->hdr.count; sfe =xfs_attr_sf_nextentry (sfe), i++) {if (xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))return -EEXIST ; }
@@ -921,6 +946,7 @@ xfs_attr_shortform_getvalue(for (i =0 ; i < sf->hdr.count; sfe =xfs_attr_sf_nextentry (sfe), i++) {if (xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))return xfs_attr_copy_value (args, &sfe->nameval[args->namelen], sfe->valuelen);
@@ -973,7 +999,7 @@ xfs_attr_shortform_to_leaf( nargs.total = args->total; nargs.whichfork =XFS_ATTR_FORK ; nargs.trans = args->trans; - nargs.op_flags =XFS_DA_OP_OKNOENT ; + nargs.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_NVLOOKUP ; sfe = &sf->list[0 ];for (i =0 ; i < sf->hdr.count; i++) {
@@ -1183,7 +1209,7 @@ xfs_attr3_leaf_to_shortform( nargs.total = args->total; nargs.whichfork =XFS_ATTR_FORK ; nargs.trans = args->trans; - nargs.op_flags =XFS_DA_OP_OKNOENT ; + nargs.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_NVLOOKUP ;for (i =0 ; i < ichdr.count; entry++, i++) {if (entry->flags &XFS_ATTR_INCOMPLETE )
@@ -2482,14 +2508,17 @@ xfs_attr3_leaf_lookup_int(if (entry->flags &XFS_ATTR_LOCAL ) { name_loc =xfs_attr3_leaf_name_local (leaf, probe);if (!xfs_attr_match (args, name_loc->namelen, - name_loc->nameval, entry->flags)) + name_loc->nameval, +be16_to_cpu (name_loc->valuelen), + &name_loc->nameval[name_loc->namelen], + entry->flags))continue ; args->index = probe;return -EEXIST ; }else { name_rmt =xfs_attr3_leaf_name_remote (leaf, probe);if (!xfs_attr_match (args, name_rmt->namelen, - name_rmt->name, entry->flags)) + name_rmt->name,0 ,NULL , entry->flags))continue ; args->index = probe; args->rmtvaluelen =be32_to_cpu (name_rmt->valuelen);
fs/xfs/libxfs/xfs_da_btree.h +3 -1
@@ -93,6 +93,7 @@ typedef struct xfs_da_args {#define XFS_DA_OP_REMOVE (1u << 6)/* this is a remove operation */ #define XFS_DA_OP_RECOVERY (1u << 7)/* Log recovery operation */ #define XFS_DA_OP_LOGGED (1u << 8)/* Use intent items to track op */ +#define XFS_DA_OP_NVLOOKUP (1u << 9)/* Match local attr on name+value */ #define XFS_DA_OP_FLAGS \ { XFS_DA_OP_JUSTCHECK, "JUSTCHECK" }, \
@@ -103,7 +104,8 @@ typedef struct xfs_da_args { { XFS_DA_OP_NOTIME, "NOTIME" }, \ { XFS_DA_OP_REMOVE, "REMOVE" }, \ { XFS_DA_OP_RECOVERY, "RECOVERY" }, \ - { XFS_DA_OP_LOGGED, "LOGGED" } + { XFS_DA_OP_LOGGED, "LOGGED" }, \ + { XFS_DA_OP_NVLOOKUP, "NVLOOKUP" }/* * Storage for holding state during Btree searches and split/join ops.
PATCH v11 4/12 xfs: preserve NVLOOKUP in xfs_attr_set
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Preserve the attr-value lookup flag when calling xfs_attr_set. Normal xattr users will never use this, but parent pointer fsck will.
--- fs/xfs/libxfs/xfs_attr.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +4 -4
@@ -986,11 +986,11 @@ xfs_attr_set(/* * We have no control over the attribute names that userspace passes us * to remove, so we have to allow the name lookup prior to attribute -* removal to fail as well. Preserve the logged flag, since we need -* to pass that through to the logging code. +* removal to fail as well. Preserve the logged and vlookup flags, +* since we need to pass them through to the lower levels. */ - args->op_flags =XFS_DA_OP_OKNOENT | - (args->op_flags &XFS_DA_OP_LOGGED ); + args->op_flags &= (XFS_DA_OP_LOGGED |XFS_DA_OP_NVLOOKUP ); + args->op_flags |=XFS_DA_OP_OKNOENT ;if (!is_remove) {XFS_STATS_INC (mp, xs_attr_set);
PATCH v11 5/12 xfs: restructure xfs_attr_complete_op a bit
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Reduce the indentation in this function by flattening the nested if statements. We're going to add more code later to this function later, hence the early cleanup. No functional changes.
--- fs/xfs/libxfs/xfs_attr.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_attr.c +5 -5
@@ -421,11 +421,11 @@ xfs_attr_complete_op(bool do_replace = args->op_flags &XFS_DA_OP_REPLACE ; args->op_flags &= ~XFS_DA_OP_REPLACE ; -if (do_replace) { - args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; -return replace_state; - } -return XFS_DAS_DONE ; +if (!do_replace) +return XFS_DAS_DONE ; + + args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; +return replace_state; } static int
PATCH v11 6/12 xfs: use helpers to extract xattr op from opflags
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create helper functions to extract the xattr op from the ondisk xattri log item and the incore attr intent item. These will get more use in the patches that follow.
--- fs/xfs/libxfs/xfs_attr.h | 5 +++++ fs/xfs/xfs_attr_item.c | 16 ++++++++++------ 2 files changed, 15 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_attr.h +5 -0
@@ -529,6 +529,11 @@ struct xfs_attr_intent {struct xfs_bmbt_irec xattri_map; }; +static inline unsigned int +xfs_attr_intent_op (const struct xfs_attr_intent *attr) +{ +return attr->xattri_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +}/*======================================================================== * Function prototypes for the kernel.
fs/xfs/xfs_attr_item.c +10 -6
@@ -351,6 +351,12 @@ xfs_xattri_finish_update(return error; } +static inline unsigned int +xfs_attr_log_item_op (const struct xfs_attri_log_format *attrp) +{ +return attrp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +} +/* Log an attr to the intent item. */ STATIC void xfs_attr_log_item(
@@ -500,8 +506,7 @@ xfs_attri_validate(struct xfs_mount *mp, struct xfs_attri_log_format *attrp) { - unsigned int op = attrp->alfi_op_flags & -XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + unsigned int op = xfs_attr_log_item_op(attrp);if (attrp->__pad !=0 )return false;
@@ -573,8 +578,7 @@ xfs_attri_item_recover( args = (struct xfs_da_args *)(attr +1 ); attr->xattri_da_args = args; - attr->xattri_op_flags = attrp->alfi_op_flags & -XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + attr->xattri_op_flags =xfs_attr_log_item_op (attrp);/* * We're reconstructing the deferred work state structure from the
@@ -596,7 +600,7 @@ xfs_attri_item_recover(ASSERT (xfs_sb_version_haslogxattrs (&mp->m_sb)); -switch (attr->xattri_op_flags) { +switch (xfs_attr_intent_op (attr)) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : args->value = nv->value.i_addr;
@@ -730,7 +734,7 @@ xlog_recover_attri_commit_pass2( }/* Check the number of log iovecs makes sense for the op code. */ - op = attri_formatp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + op =xfs_attr_log_item_op (attri_formatp);switch (op) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :
PATCH v11 7/12 xfs: validate recovered name buffers when recovering xattr items
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Strengthen the xattri log item recovery code by checking that we actually have the required name and newname buffers for whatever operation we're replaying.
--- fs/xfs/xfs_attr_item.c | 58 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 47 insertions(+), 11 deletions(-)
fs/xfs/xfs_attr_item.c +47 -11
@@ -714,22 +714,20 @@ xlog_recover_attri_commit_pass2(const void *attr_value =NULL ;const void *attr_name;size_t len; -unsigned int op; - - attri_formatp = item->ri_buf[0 ].i_addr; - attr_name = item->ri_buf[1 ].i_addr; +unsigned int op, i =0 ;/* Validate xfs_attri_log_format before the large memory allocation */ len =sizeof (struct xfs_attri_log_format ); -if (item->ri_buf[0 ].i_len != len) { +if (item->ri_buf[i].i_len != len) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len);return -EFSCORRUPTED ; } + attri_formatp = item->ri_buf[i].i_addr;if (!xfs_attri_validate (mp, attri_formatp)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len); + attri_formatp, len);return -EFSCORRUPTED ; }
@@ -758,31 +756,69 @@ xlog_recover_attri_commit_pass2( attri_formatp, len);return -EFSCORRUPTED ; } + i++;/* Validate the attr name */ -if (item->ri_buf[1 ].i_len != +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len); + attri_formatp, len);return -EFSCORRUPTED ; } + attr_name = item->ri_buf[i].i_addr;if (!xfs_attr_namecheck (attr_name, attri_formatp->alfi_name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[1 ].i_addr, item->ri_buf[1 ].i_len); + attri_formatp, len);return -EFSCORRUPTED ; } + i++;/* Validate the attr value, if present */ if (attri_formatp->alfi_value_len !=0 ) { -if (item->ri_buf[2 ].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_value_len)) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_value_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len);return -EFSCORRUPTED ; } - attr_value = item->ri_buf[2 ].i_addr; + attr_value = item->ri_buf[i].i_addr; + i++; + } + +/* +* Make sure we got the correct number of buffers for the operation +* that we just loaded. +*/ +if (i != item->ri_total) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } + +switch (op) { +case XFS_ATTRI_OP_FLAGS_REMOVE : +/* Regular remove operations operate only on names. */ +if (attr_value !=NULL || attri_formatp->alfi_value_len !=0 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } + fallthrough; +case XFS_ATTRI_OP_FLAGS_SET : +case XFS_ATTRI_OP_FLAGS_REPLACE : +/* +* Regular xattr set/remove/replace operations require a name +* and do not take a newname. Values are optional for set and +* replace. +*/ +if (attr_name ==NULL || attri_formatp->alfi_name_len ==0 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +break ; }/*
PATCH v11 8/12 xfs: always set args->value in xfs_attri_item_recover
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Always set args->value to the recovered value buffer. This reduces the amount of code in the switch statement, and hence the amount of thinking that I have to do. We validated the recovered buffers, supposedly.
--- fs/xfs/xfs_attr_item.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/xfs/xfs_attr_item.c +2 -2
@@ -594,6 +594,8 @@ xfs_attri_item_recover( args->name = nv->name.i_addr; args->namelen = nv->name.i_len; args->hashval =xfs_da_hashname (args->name, args->namelen); + args->value = nv->value.i_addr; + args->valuelen = nv->value.i_len; args->attr_filter = attrp->alfi_attr_filter &XFS_ATTRI_FILTER_MASK ; args->op_flags =XFS_DA_OP_RECOVERY |XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ;
@@ -603,8 +605,6 @@ xfs_attri_item_recover(switch (xfs_attr_intent_op (attr)) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : - args->value = nv->value.i_addr; - args->valuelen = nv->value.i_len; args->total =xfs_attr_calc_size (args, &local);if (xfs_inode_hasattr (args->dp)) attr->xattri_dela_state =xfs_attr_init_replace_state (args);
PATCH v11 9/12 xfs: use local variables for name and value length in _attri_commit_pass2
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
We're about to start using tagged unions in the xattr log format, so create a bunch of local variables in the recovery function so we only have to decode the log item fields once.
--- fs/xfs/xfs_attr_item.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
fs/xfs/xfs_attr_item.c +14 -11
@@ -711,9 +711,11 @@ xlog_recover_attri_commit_pass2(struct xfs_attri_log_item *attrip;struct xfs_attri_log_format *attri_formatp;struct xfs_attri_log_nameval *nv; -const void *attr_value =NULL ;const void *attr_name; +const void *attr_value =NULL ;size_t len; +unsigned int name_len =0 ; +unsigned int value_len =0 ;unsigned int op, i =0 ;/* Validate xfs_attri_log_format before the large memory allocation */
@@ -742,6 +744,8 @@ xlog_recover_attri_commit_pass2( attri_formatp, len);return -EFSCORRUPTED ; } + name_len = attri_formatp->alfi_name_len; + value_len = attri_formatp->alfi_value_len;break ;case XFS_ATTRI_OP_FLAGS_REMOVE :/* Log item, attr name */
@@ -750,6 +754,7 @@ xlog_recover_attri_commit_pass2( attri_formatp, len);return -EFSCORRUPTED ; } + name_len = attri_formatp->alfi_name_len;break ;default : XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
@@ -759,15 +764,14 @@ xlog_recover_attri_commit_pass2( i++;/* Validate the attr name */ -if (item->ri_buf[i].i_len != -xlog_calc_iovec_len (attri_formatp->alfi_name_len)) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ; } attr_name = item->ri_buf[i].i_addr; -if (!xfs_attr_namecheck (attr_name, attri_formatp->alfi_name_len)) { +if (!xfs_attr_namecheck (attr_name, name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -775,8 +779,8 @@ xlog_recover_attri_commit_pass2( i++;/* Validate the attr value, if present */ -if (attri_formatp->alfi_value_len !=0 ) { -if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_value_len)) { +if (value_len !=0 ) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (value_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len);
@@ -800,7 +804,7 @@ xlog_recover_attri_commit_pass2(switch (op) {case XFS_ATTRI_OP_FLAGS_REMOVE :/* Regular remove operations operate only on names. */ -if (attr_value !=NULL || attri_formatp->alfi_value_len !=0 ) { +if (attr_value !=NULL || value_len !=0 ) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -813,7 +817,7 @@ xlog_recover_attri_commit_pass2(* and do not take a newname. Values are optional for set and * replace. */ -if (attr_name ==NULL || attri_formatp->alfi_name_len ==0 ) { +if (attr_name ==NULL || name_len ==0 ) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -826,9 +830,8 @@ xlog_recover_attri_commit_pass2(* name/value buffer to the recovered incore log item and drop our * reference. */ - nv =xfs_attri_log_nameval_alloc (attr_name, - attri_formatp->alfi_name_len, attr_value, - attri_formatp->alfi_value_len); + nv =xfs_attri_log_nameval_alloc (attr_name, name_len, + attr_value, value_len); attrip =xfs_attri_init (mp, nv);memcpy (&attrip->attri_format, attri_formatp, len);
PATCH v11 10/12 xfs: log NVLOOKUP xattr removal operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If high level code wants to do a deferred xattr remove operation with the NVLOOKUP flag set, we need to push this through the log.
--- fs/xfs/libxfs/xfs_attr.c | 6 +++++- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_attr_item.c | 18 ++++++++++++++++++ 3 files changed, 24 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
@@ -942,9 +942,13 @@ xfs_attr_defer_remove( {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_REMOVE ;int error; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_REMOVE , &new); +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVREMOVE ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -957,6 +957,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_SET 1/* Set the attribute */ #define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ +#define XFS_ATTRI_OP_FLAGS_NVREMOVE 4/* Remove attr w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
fs/xfs/xfs_attr_item.c +18 -0
@@ -522,6 +522,7 @@ xfs_attri_validate(case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :case XFS_ATTRI_OP_FLAGS_REMOVE : +case XFS_ATTRI_OP_FLAGS_NVREMOVE :break ;default :return false;
@@ -611,6 +612,9 @@ xfs_attri_item_recover(else attr->xattri_dela_state =xfs_attr_init_add_state (args);break ; +case XFS_ATTRI_OP_FLAGS_NVREMOVE : + args->op_flags |=XFS_DA_OP_NVLOOKUP ; + fallthrough;case XFS_ATTRI_OP_FLAGS_REMOVE :if (!xfs_inode_hasattr (args->dp)) goto out;
@@ -736,6 +740,16 @@ xlog_recover_attri_commit_pass2(/* Check the number of log iovecs makes sense for the op code. */ op =xfs_attr_log_item_op (attri_formatp);switch (op) { +case XFS_ATTRI_OP_FLAGS_NVREMOVE : +/* Log item, attr name, optional attr value */ +if (item->ri_total !=3 && item->ri_total !=2 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } + name_len = attri_formatp->alfi_name_len; + value_len = attri_formatp->alfi_value_len; +break ;case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :/* Log item, attr name, attr value */
@@ -810,12 +824,16 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } fallthrough; +case XFS_ATTRI_OP_FLAGS_NVREMOVE :case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :/* * Regular xattr set/remove/replace operations require a name * and do not take a newname. Values are optional for set and * replace. +* +* Name-value remove operations must have a name, do not +* take a newname, and can take a value. */ if (attr_name ==NULL || name_len ==0 ) { XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
PATCH v11 11/12 xfs: log NVLOOKUP xattr setting operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If high level code wants to do a deferred xattr set operation with the NVLOOKUP flag set, we need to push this through the log.
--- fs/xfs/libxfs/xfs_attr.c | 6 +++++- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_attr_item.c | 8 +++++++- 3 files changed, 13 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
@@ -903,9 +903,13 @@ xfs_attr_defer_add( struct xfs_da_args *args) {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_SET ;int error =0 ; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_SET , &new); +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVSET ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -958,6 +958,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ #define XFS_ATTRI_OP_FLAGS_NVREMOVE 4/* Remove attr w/ vlookup */ +#define XFS_ATTRI_OP_FLAGS_NVSET 5/* Set attr with w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
fs/xfs/xfs_attr_item.c +7 -1
@@ -523,6 +523,7 @@ xfs_attri_validate(case XFS_ATTRI_OP_FLAGS_REPLACE :case XFS_ATTRI_OP_FLAGS_REMOVE :case XFS_ATTRI_OP_FLAGS_NVREMOVE : +case XFS_ATTRI_OP_FLAGS_NVSET :break ;default :return false;
@@ -604,6 +605,9 @@ xfs_attri_item_recover(ASSERT (xfs_sb_version_haslogxattrs (&mp->m_sb));switch (xfs_attr_intent_op (attr)) { +case XFS_ATTRI_OP_FLAGS_NVSET : + args->op_flags |=XFS_DA_OP_NVLOOKUP ; + fallthrough;case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : args->total =xfs_attr_calc_size (args, &local);
@@ -740,6 +744,7 @@ xlog_recover_attri_commit_pass2(/* Check the number of log iovecs makes sense for the op code. */ op =xfs_attr_log_item_op (attri_formatp);switch (op) { +case XFS_ATTRI_OP_FLAGS_NVSET :case XFS_ATTRI_OP_FLAGS_NVREMOVE :/* Log item, attr name, optional attr value */ if (item->ri_total !=3 && item->ri_total !=2 ) {
@@ -824,6 +829,7 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } fallthrough; +case XFS_ATTRI_OP_FLAGS_NVSET :case XFS_ATTRI_OP_FLAGS_NVREMOVE :case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :
@@ -832,7 +838,7 @@ xlog_recover_attri_commit_pass2(* and do not take a newname. Values are optional for set and * replace. * -* Name-value remove operations must have a name, do not +* Name-value set/remove operations must have a name, do not * take a newname, and can take a value. */ if (attr_name ==NULL || name_len ==0 ) {
PATCH v11 12/12 xfs: log NVLOOKUP xattr nvreplace operations
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
(Formerly titled "xfs: Add new name to attri/d" and described as follows:
This patch adds two new fields to the atti/d. They are nname and nnamelen. This will be used for parent pointer updates since a rename operation may cause the parent pointer to update both the name and value. So we need to carry both the new name as well as the target name in the attri/d.)
If high level code wants to do a deferred xattr nvreplace operation with the NVLOOKUP flag set, we need to push this through the log. This log item records the old name/value pair and the new name/value pair, and completely replaces one with the other. Parent pointers will need this ability to handle rename moving a child file between parents.
--- fs/xfs/libxfs/xfs_attr.c | 20 ++++ fs/xfs/libxfs/xfs_attr.h | 4 - fs/xfs/libxfs/xfs_da_btree.h | 6 + fs/xfs/libxfs/xfs_log_format.h | 28 ++++- fs/xfs/xfs_attr_item.c | 207 +++++++++++++++++++++++++++++++++++----- fs/xfs/xfs_attr_item.h | 2 6 files changed, 233 insertions(+), 34 deletions(-)
fs/xfs/libxfs/xfs_attr.c +19 -1
@@ -425,6 +425,20 @@ xfs_attr_complete_op(return XFS_DAS_DONE ; args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; +if (xfs_attr_intent_op (attr) !=XFS_ATTRI_OP_FLAGS_NVREPLACE ) +return replace_state; + +/* +* NVREPLACE operations require the caller to set the old and new names +* and values explicitly. +*/ +ASSERT (args->new_namelen >0 ); + + args->name = args->new_name; + args->namelen = args->new_namelen; + args->hashval =xfs_da_hashname (args->name, args->namelen); + args->value = args->new_value; + args->valuelen = args->new_valuelen;return replace_state; }
@@ -926,9 +940,13 @@ xfs_attr_defer_replace( struct xfs_da_args *args) {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_REPLACE ;int error =0 ; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_REPLACE , &new); +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVREPLACE ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
fs/xfs/libxfs/xfs_attr.h +2 -2
@@ -510,8 +510,8 @@ struct xfs_attr_intent {struct xfs_da_args *xattri_da_args;/* -* Shared buffer containing the attr name and value so that the logging -* code can share large memory buffers between log items. +* Shared buffer containing the attr name, new name, and value so that +* the logging code can share large memory buffers between log items. */ struct xfs_attri_log_nameval *xattri_nameval;
fs/xfs/libxfs/xfs_da_btree.h +5 -1
@@ -54,11 +54,15 @@ enum xfs_dacmp {*/ typedef struct xfs_da_args { struct xfs_da_geometry *geo; /* da block geometry */ - const uint8_t *name; /* string (maybe not NULL terminated) */ + const uint8_t *name; /* string (maybe not NULL terminated) */ + const uint8_t *new_name; /* new attr name */ int namelen; /* length of string (maybe no NULL) */ + int new_namelen; /* new attr name len */ uint8_t filetype; /* filetype of inode for directories */ void *value; /* set of bytes (maybe contain NULLs) */ + void *new_value; /* new xattr value (may contain NULLs) */ int valuelen; /* length of value */ + int new_valuelen; /* length of new attr value */ unsigned int attr_filter; /* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ unsigned int attr_flags; /* XATTR_{CREATE,REPLACE} */ xfs_dahash_t hashval; /* hash value of name */
fs/xfs/libxfs/xfs_log_format.h +23 -5
@@ -115,10 +115,11 @@ struct xfs_unmount_log_format {#define XLOG_REG_TYPE_BUD_FORMAT 26#define XLOG_REG_TYPE_ATTRI_FORMAT 27#define XLOG_REG_TYPE_ATTRD_FORMAT 28 -#define XLOG_REG_TYPE_ATTR_NAME 29 +#define XLOG_REG_TYPE_ATTR_NAME 29#define XLOG_REG_TYPE_ATTR_VALUE 30 -#define XLOG_REG_TYPE_MAX 30 - +#define XLOG_REG_TYPE_ATTR_NEWNAME 31 +#define XLOG_REG_TYPE_ATTR_NEWVALUE 32 +#define XLOG_REG_TYPE_MAX 32/* * Flags to log operation header
@@ -959,6 +960,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ #define XFS_ATTRI_OP_FLAGS_NVREMOVE 4/* Remove attr w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_NVSET 5/* Set attr with w/ vlookup */ +#define XFS_ATTRI_OP_FLAGS_NVREPLACE 6/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
@@ -976,11 +978,27 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t __pad;/* pad to 64 bit aligned */ + +/* +* For NVREPLACE, this is the length of the new xattr value. +* alfi_value_len contains the length of the old xattr value. +*/ +uint32_t alfi_new_value_len; +uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */ uint32_t alfi_op_flags;/* marks the op as a set or remove */ -uint32_t alfi_name_len;/* attr name length */ +union { +uint32_t alfi_name_len;/* attr name length */ +struct { +/* +* For NVREPLACE, these are the lengths of the old and +* new attr name. +*/ +uint16_t alfi_old_name_len; +uint16_t alfi_new_name_len; + }; + };uint32_t alfi_value_len;/* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */ };
fs/xfs/xfs_attr_item.c +182 -25
352 lines not on this page · raw message
fs/xfs/xfs_attr_item.h +2 -0
@@ -13,7 +13,9 @@ struct kmem_zone;struct xfs_attri_log_nameval {struct xfs_log_iovec name; +struct xfs_log_iovec new_name;/* NVREPLACE only */ struct xfs_log_iovec value; +struct xfs_log_iovec new_value;/* NVREPLACE only */ refcount_t refcount;/* name and value follow the end of this struct */
v11 cover xfs: hold ILOCK during deferred dir ops
In reply to Darrick J. WongHi all,
Currently, directory updates (link, unlink, rename) only hold the ILOCK for the first transaction in the chain. The first transaction is where we perform all the directory updates, so this has sufficed to coordinate access to the directory itself.
With parent pointers, we need to hold both directories and children ILOCKed across the entire directory update transaction chain so that other threads never see an inconsistent edge state (parent -> child and parent <- child). Prepare for this by making the directory update code hold all the ILOCKs.
There's a subtle issue with online rmapbt repair that gets fixed here. Space allocations performed as part of a directory update result in deferred rmap updates later in the chain. With the current code, the directory ILOCK (but not the IOLOCK) is dropped before the rmapbt updates are performed. As a result, the online rmapbt repair scanner has to hold each directory's IOLOCK and ILOCK to coordinate with writer threads correctly. This change makes it so that online repair no longer has to hold the directory IOLOCK, which makes the locking model here consistent with the other repair scanners.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-hold-ilocks
--- fs/xfs/xfs_inode.c | 65 ++++++++++++++++++++++++++++++++++++++------------ fs/xfs/xfs_qm.c | 4 ++- fs/xfs/xfs_symlink.c | 2 ++ fs/xfs/xfs_trans.c | 9 +++++-- 4 files changed, 61 insertions(+), 19 deletions(-)
PATCH v11 1/3 xfs: Hold inode locks in xfs_ialloc
From: Allison Henderson <[email protected]>
Modify xfs_ialloc to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 8 +++++++- fs/xfs/xfs_qm.c | 4 +++- fs/xfs/xfs_symlink.c | 2 ++ 3 files changed, 12 insertions(+), 2 deletions(-)
fs/xfs/xfs_inode.c +7 -1
@@ -774,6 +774,8 @@ xfs_inode_inherit_flags2(/* * Initialise a newly allocated inode and return the in-core inode to the * caller locked exclusively. +* +* Caller is responsible for unlocking the inode manually upon return */ int xfs_init_new_inode(
@@ -899,7 +901,7 @@ xfs_init_new_inode(/* * Log the new values stuffed into the inode. */ -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, ip,0 );xfs_trans_log_inode (tp, ip, flags);/* now that we have an i_mode we can setup the inode structure */
@@ -1076,6 +1078,7 @@ xfs_create(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -1087,6 +1090,7 @@ xfs_create(* transactions and deadlocks from xfs_inactive. */ if (ip) { +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_finish_inode_setup (ip);xfs_irele (ip); }
@@ -1172,6 +1176,7 @@ xfs_create_tmpfile(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -1183,6 +1188,7 @@ xfs_create_tmpfile(* transactions and deadlocks from xfs_inactive. */ if (ip) { +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_finish_inode_setup (ip);xfs_irele (ip); }
fs/xfs/xfs_qm.c +3 -1
@@ -826,8 +826,10 @@ xfs_qm_qino_alloc(ASSERT (xfs_is_shutdown (mp));xfs_alert (mp,"%s failed (error %d)!" , __func__, error); } -if (need_alloc) +if (need_alloc) {xfs_finish_inode_setup (*ipp); +xfs_iunlock (*ipp,XFS_ILOCK_EXCL ); + }return error; }
fs/xfs/xfs_symlink.c +2 -0
@@ -337,6 +337,7 @@ xfs_symlink(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -348,6 +349,7 @@ xfs_symlink(* transactions and deadlocks from xfs_inactive. */ if (ip) { +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_finish_inode_setup (ip);xfs_irele (ip); }
PATCH v11 2/3 xfs: Hold inode locks in xfs_trans_alloc_dir
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_trans_alloc_dir to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 14 ++++++++++++-- fs/xfs/xfs_trans.c | 9 +++++++-- 2 files changed, 19 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +12 -2
@@ -1279,10 +1279,15 @@ xfs_link(if (xfs_has_wsync (mp) ||xfs_has_dirsync (mp))xfs_trans_set_sync (tp); -return xfs_trans_commit (tp); + error =xfs_trans_commit (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); +return error; error_return:xfs_trans_cancel (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;
@@ -2518,15 +2523,20 @@ xfs_remove( error =xfs_trans_commit (tp);if (error) - goto std_return; + goto out_unlock;if (is_dir &&xfs_inode_is_filestream (ip))xfs_filestream_deassociate (ip); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:xfs_trans_cancel (tp); + out_unlock: +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL ); std_return:return error; }
fs/xfs/xfs_trans.c +7 -2
@@ -1356,6 +1356,8 @@ xfs_trans_alloc_ichange(* The caller must ensure that the on-disk dquots attached to this inode have * already been allocated and initialized. The ILOCKs will be dropped when the * transaction is committed or cancelled. +* +* Caller is responsible for unlocking the inodes manually upon return */ int xfs_trans_alloc_dir(
@@ -1386,8 +1388,8 @@ xfs_trans_alloc_dir(xfs_lock_two_inodes (dp,XFS_ILOCK_EXCL , ip,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); +xfs_trans_ijoin (tp, ip,0 ); error =xfs_qm_dqattach_locked (dp, false);if (error) {
@@ -1410,6 +1412,9 @@ xfs_trans_alloc_dir(if (error == -EDQUOT || error == -ENOSPC ) {if (!retried) {xfs_trans_cancel (tp); +xfs_iunlock (dp,XFS_ILOCK_EXCL ); +if (dp != ip) +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_blockgc_free_quota (dp,0 ); retried = true; goto retry;
PATCH v11 3/3 xfs: Hold inode locks in xfs_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_rename to hold all inode locks across a rename operation We will need this later when we add parent pointers
--- fs/xfs/xfs_inode.c | 43 ++++++++++++++++++++++++++++++------------- 1 file changed, 30 insertions(+), 13 deletions(-)
fs/xfs/xfs_inode.c +30 -13
@@ -2541,6 +2541,21 @@ xfs_remove(return error; } +static inline void +xfs_iunlock_rename ( +struct xfs_inode **i_tab, +int num_inodes) +{ +int i; + +for (i = num_inodes -1 ; i >=0 ; i--) { +/* Skip duplicate inodes if src and target dps are the same */ +if (!i_tab[i] || (i >0 && i_tab[i] == i_tab[i -1 ])) +continue ; +xfs_iunlock (i_tab[i],XFS_ILOCK_EXCL ); + } +} +/* * Enter all inodes for a rename transaction into a sorted array. */
@@ -2839,18 +2854,16 @@ xfs_rename(xfs_lock_inodes (inodes, num_inodes,XFS_ILOCK_EXCL );/* -* Join all the inodes to the transaction. From this point on, -* we can rely on either trans_commit or trans_cancel to unlock -* them. +* Join all the inodes to the transaction. */ -xfs_trans_ijoin (tp, src_dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, src_dp,0 );if (new_parent) -xfs_trans_ijoin (tp, target_dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, src_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_dp,0 ); +xfs_trans_ijoin (tp, src_ip,0 );if (target_ip) -xfs_trans_ijoin (tp, target_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_ip,0 );if (wip) -xfs_trans_ijoin (tp, wip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, wip,0 );/* * If we are using project inheritance, we only allow renames
@@ -2864,10 +2877,12 @@ xfs_rename( }/* RENAME_EXCHANGE is unique from here on. */ -if (flags &RENAME_EXCHANGE ) -return xfs_cross_rename (tp, src_dp, src_name, src_ip, +if (flags &RENAME_EXCHANGE ) { + error =xfs_cross_rename (tp, src_dp, src_name, src_ip, target_dp, target_name, target_ip, spaceres); + goto out_unlock; + }/* * Try to reserve quota to handle an expansion of the target directory.
@@ -2881,6 +2896,7 @@ xfs_rename(if (error == -EDQUOT || error == -ENOSPC ) {if (!retried) {xfs_trans_cancel (tp); +xfs_iunlock_rename (inodes, num_inodes);xfs_blockgc_free_quota (target_dp,0 ); retried = true; goto retry;
@@ -3092,12 +3108,13 @@ xfs_rename(xfs_trans_log_inode (tp, target_dp,XFS_ILOG_CORE ); error =xfs_finish_rename (tp); -if (wip) -xfs_irele (wip); -return error; + + goto out_unlock; out_trans_cancel:xfs_trans_cancel (tp); +out_unlock: +xfs_iunlock_rename (inodes, num_inodes); out_release_wip:if (wip)xfs_irele (wip);
v11 cover xfs: Parent Pointers
In reply to Darrick J. WongHi all,
This is the latest parent pointer attributes for xfs.
The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point.
Updates since v10 [djwong]:
Merge in the ondisk format changes to get rid of the diroffset conflicts with the parent pointer repair code, rebase the entire series with the attr vlookup changes first, and merge all the other random fixes.
Updates since v9:
Reordered patches 2 and 3 to be 6 and 7
xfs: Add xfs_verify_pptr
moved parent pointer validators to xfs_parent
xfs: Add parent pointer ioctl
Extra validation checks for fs id
added missing release for the inode
use GFP_KERNEL flags for malloc/realloc reworked ioctl to use pptr listenty and flex array
NEW
xfs: don't remove the attr fork when parent pointers are enabled
NEW
directory lookups should return diroffsets too
NEW
xfs: move/add parent pointer validators to xfs_parent
Updates since v8:
xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec
xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas
xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates
xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free
xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once Fix xfs_rename to disallow reservationless quotas Fix double unlock on dqattach fail
Move parent frees to out_release_wip
xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename
Questions comments and feedback appreciated!
Thanks all!
Allison
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_attr.c | 21 ++ fs/xfs/libxfs/xfs_attr.h | 14 +- fs/xfs/libxfs/xfs_attr_leaf.c | 6 - fs/xfs/libxfs/xfs_attr_sf.h | 1 fs/xfs/libxfs/xfs_da_format.h | 34 ++++ fs/xfs/libxfs/xfs_defer.c | 28 +++ fs/xfs/libxfs/xfs_defer.h | 8 + fs/xfs/libxfs/xfs_format.h | 4 fs/xfs/libxfs/xfs_fs.h | 2 fs/xfs/libxfs/xfs_fs_staging.h | 84 ++++++++++ fs/xfs/libxfs/xfs_log_format.h | 1 fs/xfs/libxfs/xfs_log_rlimit.c | 53 ++++++ fs/xfs/libxfs/xfs_parent.c | 317 ++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 104 ++++++++++++ fs/xfs/libxfs/xfs_sb.c | 4 fs/xfs/libxfs/xfs_trans_resv.c | 326 +++++++++++++++++++++++++++++++------ fs/xfs/libxfs/xfs_trans_space.h | 8 - fs/xfs/scrub/attr.c | 12 + fs/xfs/xfs_attr_item.c | 24 ++- fs/xfs/xfs_attr_list.c | 25 ++- fs/xfs/xfs_dquot.c | 38 ++++ fs/xfs/xfs_dquot.h | 1 fs/xfs/xfs_file.c | 1 fs/xfs/xfs_inode.c | 343 ++++++++++++++++++++++++++++++++------- fs/xfs/xfs_inode.h | 9 + fs/xfs/xfs_ioctl.c | 189 ++++++++++++++++++++- fs/xfs/xfs_ioctl.h | 2 fs/xfs/xfs_iops.c | 2 fs/xfs/xfs_linux.h | 1 fs/xfs/xfs_ondisk.h | 4 fs/xfs/xfs_parent_utils.c | 157 ++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 20 ++ fs/xfs/xfs_qm.h | 2 fs/xfs/xfs_super.c | 14 ++ fs/xfs/xfs_symlink.c | 49 +++++- fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 76 +++++++++ fs/xfs/xfs_trans_dquot.c | 15 +- fs/xfs/xfs_xattr.c | 8 + fs/xfs/xfs_xattr.h | 2 41 files changed, 1829 insertions(+), 183 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_fs_staging.h create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h
PATCH v11 1/23 xfs: Expose init_xattrs in xfs_create_tmpfile
From: Allison Henderson <[email protected]>
Tmp files are used as part of rename operations and will need attr forks initialized for parent pointers. Expose the init_xattrs parameter to the calling function to initialize the fork.
--- fs/xfs/xfs_inode.c | 5 +++-- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_iops.c | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +3 -2
@@ -1109,6 +1109,7 @@ xfs_create_tmpfile(struct mnt_idmap *idmap, struct xfs_inode *dp, umode_t mode, + bool init_xattrs, struct xfs_inode **ipp) { struct xfs_mount *mp = dp->i_mount;
@@ -1149,7 +1150,7 @@ xfs_create_tmpfile( error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error =xfs_init_new_inode (idmap, tp, dp, ino, mode, -0 ,0 , prid, false, &ip); +0 ,0 , prid, init_xattrs, &ip);if (error) goto out_trans_cancel;
@@ -2750,7 +2751,7 @@ xfs_rename_alloc_whiteout(int error; error =xfs_create_tmpfile (idmap, dp,S_IFCHR |WHITEOUT_MODE , - &tmpfile); + false, &tmpfile);if (error)return error;
fs/xfs/xfs_inode.h +1 -1
@@ -478,7 +478,7 @@ int xfs_create(struct mnt_idmap *idmap, umode_t mode, dev_t rdev, bool need_xattr, struct xfs_inode **ipp);int xfs_create_tmpfile (struct mnt_idmap *idmap, -struct xfs_inode *dp,umode_t mode, +struct xfs_inode *dp,umode_t mode,bool init_xattrs,struct xfs_inode **ipp);int xfs_remove (struct xfs_inode *dp,struct xfs_name *name,struct xfs_inode *ip);
fs/xfs/xfs_iops.c +1 -1
@@ -200,7 +200,7 @@ xfs_generic_create(xfs_create_need_xattr (dir, default_acl, acl), &ip); }else { - error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, &ip); + error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, true, &ip); }if (unlikely (error)) goto out_free_acl;
PATCH v11 2/23 xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Renames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.
--- fs/xfs/libxfs/xfs_defer.c | 28 ++++++++++++++++++++++++++-- fs/xfs/libxfs/xfs_defer.h | 8 +++++++- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 1 + 4 files changed, 35 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_defer.c +26 -2
@@ -820,13 +820,37 @@ xfs_defer_ops_continue(struct xfs_trans *tp, struct xfs_defer_resources *dres) { - unsigned int i; + unsigned int i, j; +struct xfs_inode *sips[XFS_DEFER_OPS_NR_INODES ]; +struct xfs_inode *temp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );ASSERT (!(tp->t_flags &XFS_TRANS_DIRTY ));/* Lock the captured resources to the new transaction. */ -if (dfc->dfc_held.dr_inos ==2 ) +if (dfc->dfc_held.dr_inos >2 ) { +/* +* Renames with parent pointer updates can lock up to 5 inodes, +* sorted by their inode number. So we need to make sure they +* are relocked in the same way. +*/ +memset (sips,0 ,sizeof (sips)); +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) + sips[i] = dfc->dfc_held.dr_ip[i]; + +/* Bubble sort of at most 5 inodes */ +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) { +for (j =1 ; j < dfc->dfc_held.dr_inos; j++) { +if (sips[j]->i_ino < sips[j-1 ]->i_ino) { + temp = sips[j]; + sips[j] = sips[j-1 ]; + sips[j-1 ] = temp; + } + } + } + +xfs_lock_inodes (sips, dfc->dfc_held.dr_inos,XFS_ILOCK_EXCL ); + }else if (dfc->dfc_held.dr_inos ==2 )xfs_lock_two_inodes (dfc->dfc_held.dr_ip[0 ],XFS_ILOCK_EXCL , dfc->dfc_held.dr_ip[1 ],XFS_ILOCK_EXCL );else if (dfc->dfc_held.dr_inos ==1 )
fs/xfs/libxfs/xfs_defer.h +7 -1
@@ -70,7 +70,13 @@ extern const struct xfs_defer_op_type xfs_attr_defer_type;/* * Deferred operation item relogging limits. */ -#define XFS_DEFER_OPS_NR_INODES 2/* join up to two inodes */ + +/* +* Rename w/ parent pointers can require up to 5 inodes with deferred ops to +* be joined to the transaction: src_dp, target_dp, src_ip, target_ip, and wip. +* These inodes are locked in sorted order by their inode numbers +*/ +#define XFS_DEFER_OPS_NR_INODES 5#define XFS_DEFER_OPS_NR_BUFS 2/* join up to two buffers */ /* Resources that must be held across a transaction roll. */
fs/xfs/xfs_inode.c +1 -1
@@ -447,7 +447,7 @@ xfs_lock_inumorder(* lock more than one at a time, lockdep will report false positives saying we * have violated locking orders. */ -static void +void xfs_lock_inodes( struct xfs_inode **ips, int inodes,
fs/xfs/xfs_inode.h +1 -0
@@ -574,5 +574,6 @@ void xfs_end_io(struct work_struct *work);int xfs_ilock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_iunlock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2); +void xfs_lock_inodes (struct xfs_inode **ips,int inodes,uint lock_mode);#endif /* __XFS_INODE_H__ */
PATCH v11 3/23 xfs: Increase XFS_QM_TRANS_MAXDQS to 5
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
With parent pointers enabled, a rename operation can update up to 5
inodes: src_dp, target_dp, src_ip, target_ip and wip. This causes their dquots to a be attached to the transaction chain, so we need to increase XFS_QM_TRANS_MAXDQS. This patch also add a helper function xfs_dqlockn to lock an arbitrary number of dquots.
--- fs/xfs/xfs_dquot.c | 38 ++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_dquot.h | 1 + fs/xfs/xfs_qm.h | 2 +- fs/xfs/xfs_trans_dquot.c | 15 ++++++++++----- 4 files changed, 50 insertions(+), 6 deletions(-)
fs/xfs/xfs_dquot.c +38 -0
@@ -1333,6 +1333,44 @@ xfs_dqlock2( } } +static int +xfs_dqtrx_cmp ( +const void *a, +const void *b) +{ +const struct xfs_dqtrx *qa = a; +const struct xfs_dqtrx *qb = b; + +if (qa->qt_dquot->q_id > qb->qt_dquot->q_id) +return 1 ; +if (qa->qt_dquot->q_id < qb->qt_dquot->q_id) +return -1 ; +return 0 ; +} + +void +xfs_dqlockn ( +struct xfs_dqtrx *q) +{ +unsigned int i; + +/* Sort in order of dquot id, do not allow duplicates */ +for (i =0 ; i <XFS_QM_TRANS_MAXDQS && q[i].qt_dquot !=NULL ; i++) { +unsigned int j; + +for (j =0 ; j < i; j++) +ASSERT (q[i].qt_dquot != q[j].qt_dquot); + } +if (i ==0 ) +return ; + +sort (q, i,sizeof (struct xfs_dqtrx ), xfs_dqtrx_cmp,NULL ); + +mutex_lock (&q[0 ].qt_dquot->q_qlock); +for (i =1 ; i <XFS_QM_TRANS_MAXDQS && q[i].qt_dquot !=NULL ; i++) +mutex_lock_nested (&q[i].qt_dquot->q_qlock,XFS_QLOCK_NESTED ); +} +int __init xfs_qm_init (void ) {
fs/xfs/xfs_dquot.h +1 -0
@@ -223,6 +223,7 @@ int xfs_qm_dqget_uncached(struct xfs_mount *mp,void xfs_qm_dqput (struct xfs_dquot *dqp);void xfs_dqlock2 (struct xfs_dquot *,struct xfs_dquot *); +void xfs_dqlockn (struct xfs_dqtrx *q);void xfs_dquot_set_prealloc_limits (struct xfs_dquot *);
fs/xfs/xfs_qm.h +1 -1
@@ -120,7 +120,7 @@ enum {XFS_QM_TRANS_PRJ , XFS_QM_TRANS_DQTYPES }; -#define XFS_QM_TRANS_MAXDQS 2 +#define XFS_QM_TRANS_MAXDQS 5struct xfs_dquot_acct {struct xfs_dqtrx dqs[XFS_QM_TRANS_DQTYPES ][XFS_QM_TRANS_MAXDQS ]; };
fs/xfs/xfs_trans_dquot.c +10 -5
@@ -268,24 +268,29 @@ xfs_trans_mod_dquot(/* * Given an array of dqtrx structures, lock all the dquots associated and join -* them to the transaction, provided they have been modified. We know that the -* highest number of dquots of one type - usr, grp and prj - involved in a -* transaction is 3 so we don't need to make this very generic. +* them to the transaction, provided they have been modified. */ STATIC voidxfs_trans_dqlockedjoin (struct xfs_trans *tp,struct xfs_dqtrx *q) { +unsigned int i;ASSERT (q[0 ].qt_dquot !=NULL );if (q[1 ].qt_dquot ==NULL ) {xfs_dqlock (q[0 ].qt_dquot);xfs_trans_dqjoin (tp, q[0 ].qt_dquot); - }else { -ASSERT (XFS_QM_TRANS_MAXDQS ==2 ); + }else if (q[2 ].qt_dquot ==NULL ) {xfs_dqlock2 (q[0 ].qt_dquot, q[1 ].qt_dquot);xfs_trans_dqjoin (tp, q[0 ].qt_dquot);xfs_trans_dqjoin (tp, q[1 ].qt_dquot); + }else { +xfs_dqlockn (q); +for (i =0 ; i <XFS_QM_TRANS_MAXDQS ; i++) { +if (q[i].qt_dquot ==NULL ) +break ; +xfs_trans_dqjoin (tp, q[i].qt_dquot); + } } }
PATCH v11 4/23 xfs: add parent pointer support to attribute code
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer entries; it uses reserved blocks like XFS_ATTR_ROOT.
--- fs/xfs/libxfs/xfs_attr.c | 3 ++- fs/xfs/libxfs/xfs_da_format.h | 5 ++++- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_trace.h | 3 ++- 5 files changed, 10 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +2 -1
@@ -992,7 +992,8 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; -bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool rsvd = (args->attr_filter & (XFS_ATTR_ROOT | +XFS_ATTR_PARENT ));bool is_remove = args->op_flags &XFS_DA_OP_REMOVE ;int error, local;int rmt_blks =0 ;
fs/xfs/libxfs/xfs_da_format.h +4 -1
@@ -688,12 +688,15 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */ #define XFS_ATTR_SECURE_BIT 2/* limit access to secure attrs */ +#define XFS_ATTR_PARENT_BIT 3/* parent pointer attrs */ #define XFS_ATTR_INCOMPLETE_BIT 7/* attr in middle of create/delete */ #define XFS_ATTR_LOCAL (1u << XFS_ATTR_LOCAL_BIT)#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT) +#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_NSP_ONDISK_MASK \ + (XFS_ATTR_ROOT | XFS_ATTR_SECURE | XFS_ATTR_PARENT)/* * Alignment for namelist and valuelist entries (since they are mixed
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -969,6 +969,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/*
fs/xfs/scrub/attr.c +1 -1
@@ -441,7 +441,7 @@ xchk_xattr_rec(/* Retrieve the entry and check it. */ hash =be32_to_cpu (ent->hashval); badflags = ~(XFS_ATTR_LOCAL |XFS_ATTR_ROOT |XFS_ATTR_SECURE | -XFS_ATTR_INCOMPLETE ); +XFS_ATTR_INCOMPLETE |XFS_ATTR_PARENT );if ((ent->flags & badflags) !=0 )xchk_da_set_corrupt (ds, level);if (ent->flags &XFS_ATTR_LOCAL ) {
fs/xfs/xfs_trace.h +2 -1
@@ -79,7 +79,8 @@ struct xfs_perag;#define XFS_ATTR_FILTER_FLAGS \ { XFS_ATTR_ROOT, "ROOT" }, \ { XFS_ATTR_SECURE, "SECURE" }, \ - { XFS_ATTR_INCOMPLETE, "INCOMPLETE" } + { XFS_ATTR_INCOMPLETE, "INCOMPLETE" }, \ + { XFS_ATTR_PARENT, "PARENT" } DECLARE_EVENT_CLASS(xfs_attr_list_class, TP_PROTO(struct xfs_attr_list_context *ctx),
PATCH v11 5/23 xfs: define parent pointer ondisk extended attribute format
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to define the parent pointer attribute format before we start adding support for it into all the code that needs to use it. The EA format we will use encodes the following information:
name={parent inode #, parent inode generation, dirent namehash}
value={dirent name}The inode/gen gives all the information we need to reliably identify the parent without requiring child->parent lock ordering, and allows userspace to do pathname component level reconstruction without the kernel ever needing to verify the parent itself as part of ioctl calls. Storing the dirent name hash in the key reduces hash collisions if a file is hardlinked multiple times in the same directory.
By using the NVLOOKUP mode in the extended attribute code to match parent pointers using both the xattr name and value, we can identify the exact parent pointer EA we need to modify/remove in rename/unlink operations without searching the entire EA space.
By storing the dirent name, we have enough information to be able to validate and reconstruct damaged directory trees. Earlier iterations of this patchset encoded the directory offset in the parent pointer key, but this format required repair to keep that in sync across directory rebuilds, which is unnecessary complexity.
--- fs/xfs/libxfs/xfs_da_format.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
fs/xfs/libxfs/xfs_da_format.h +21 -0
@@ -805,4 +805,25 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); +/* +* Parent pointer attribute format definition +* +* The xattr name encodes the parent inode number, generation and the crc32c +* hash of the dirent name. +* +* The xattr value contains the dirent name. +*/ +struct xfs_parent_name_rec { +__be64 p_ino; +__be32 p_gen; +__be32 p_namehash; +}; + +/* +* Maximum size of the dirent name that can be stored in a parent pointer. +* This matches the maximum dirent name length. +*/ +#define XFS_PARENT_DIRENT_NAME_MAX_SIZE \ + (MAXNAMELEN - 1) +#endif /* __XFS_DA_FORMAT_H__ */
PATCH v11 6/23 xfs: add parent pointer validator functions
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings. So we need to modify attr_namecheck to verify parent pointer records when the XFS_ATTR_PARENT flag is set. At the same time, we need to validate attr values during log recovery if the xattr is really a parent pointer.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_attr.c | 10 ++++ fs/xfs/libxfs/xfs_attr.h | 3 + fs/xfs/libxfs/xfs_da_format.h | 8 ++++ fs/xfs/libxfs/xfs_parent.c | 92 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 16 +++++++ fs/xfs/scrub/attr.c | 2 - fs/xfs/xfs_attr_item.c | 24 +++++++++-- fs/xfs/xfs_attr_list.c | 17 +++++--- 9 files changed, 161 insertions(+), 12 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h
fs/xfs/Makefile +1 -0
@@ -41,6 +41,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_buf.o \ xfs_log_rlimit.o \ xfs_ag_resv.o \ + xfs_parent.o \ xfs_rmap.o \ xfs_rmap_btree.o \ xfs_refcount.o \
fs/xfs/libxfs/xfs_attr.c +8 -2
@@ -26,6 +26,7 @@#include "xfs_trace.h" #include "xfs_attr_item.h" #include "xfs_xattr.h" +#include "xfs_parent.h" struct kmem_cache *xfs_attr_intent_cache;
@@ -1596,9 +1597,14 @@ xfs_attr_node_get(/* Returns true if the attribute entry name is valid. */ bool xfs_attr_namecheck ( -const void *name, -size_t length) +struct xfs_mount *mp, +const void *name, +size_t length, +unsigned int flags) { +if (flags &XFS_ATTR_PARENT ) +return xfs_parent_namecheck (mp, name, length, flags); +/* * MAXNAMELEN includes the trailing null, but (name/length) leave it * out, so use >= for the length check.
fs/xfs/libxfs/xfs_attr.h +2 -1
@@ -552,7 +552,8 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, +unsigned int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);
fs/xfs/libxfs/xfs_da_format.h +8 -0
@@ -731,6 +731,14 @@ xfs_attr3_leaf_name(xfs_attr_leafblock_t *leafp, int idx)return &((char *)leafp)[be16_to_cpu (entries[idx].nameidx)]; } +static inline int +xfs_attr3_leaf_flags (xfs_attr_leafblock_t *leafp,int idx) +{ +struct xfs_attr_leaf_entry *entries =xfs_attr3_leaf_entryp (leafp); + +return entries[idx].flags; +} +static inline xfs_attr_leaf_name_remote_t *xfs_attr3_leaf_name_remote (xfs_attr_leafblock_t *leafp,int idx) {
fs/xfs/libxfs/xfs_parent.c new file +92 -0
92 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022-2023 Oracle, Inc. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* Metadata validators */ +bool xfs_parent_namecheck (struct xfs_mount *mp, +const struct xfs_parent_name_rec *rec,size_t reclen, +unsigned int attr_flags); +bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value, +size_t valuelen); + +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/attr.c +1 -1
@@ -129,7 +129,7 @@ xchk_xattr_listent( }/* Does this name make sense? */ -if (!xfs_attr_namecheck (name, namelen)) { +if (!xfs_attr_namecheck (sx->sc->mp, name, namelen, flags)) {xchk_fblock_set_corrupt (sx->sc,XFS_ATTR_FORK , args.blkno);return ; }
fs/xfs/xfs_attr_item.c +21 -3
@@ -27,6 +27,7 @@#include "xfs_error.h" #include "xfs_log_priv.h" #include "xfs_log_recover.h" +#include "xfs_parent.h" struct kmem_cache *xfs_attri_cache;struct kmem_cache *xfs_attrd_cache;
@@ -644,7 +645,8 @@ xfs_attri_item_recover(*/ attrp = &attrip->attri_format;if (!xfs_attri_validate (mp, attrp) || - !xfs_attr_namecheck (nv->name.i_addr, nv->name.i_len)) + !xfs_attr_namecheck (mp, nv->name.i_addr, nv->name.i_len, + attrp->alfi_attr_filter))return -EFSCORRUPTED ; error =xlog_recover_iget (mp, attrp->alfi_ino, &ip);
@@ -898,7 +900,8 @@ xlog_recover_attri_commit_pass2( } attr_name = item->ri_buf[i].i_addr; -if (!xfs_attr_namecheck (attr_name, name_len)) { +if (!xfs_attr_namecheck (mp, attr_name, name_len, + attri_formatp->alfi_attr_filter)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -915,7 +918,8 @@ xlog_recover_attri_commit_pass2( } attr_new_name = item->ri_buf[i].i_addr; -if (!xfs_attr_namecheck (attr_new_name, new_name_len)) { +if (!xfs_attr_namecheck (mp, attr_new_name, new_name_len, + attri_formatp->alfi_attr_filter)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[i].i_addr, item->ri_buf[i].i_len);
@@ -933,6 +937,13 @@ xlog_recover_attri_commit_pass2( } attr_value = item->ri_buf[i].i_addr; +if ((attri_formatp->alfi_attr_filter &XFS_ATTR_PARENT ) && + !xfs_parent_valuecheck (mp, attr_value, value_len)) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + item->ri_buf[i].i_addr, + item->ri_buf[i].i_len); +return -EFSCORRUPTED ; + } i++; }
@@ -945,6 +956,13 @@ xlog_recover_attri_commit_pass2( } attr_new_value = item->ri_buf[i].i_addr; +if ((attri_formatp->alfi_attr_filter &XFS_ATTR_PARENT ) && + !xfs_parent_valuecheck (mp, attr_new_value, new_value_len)) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + item->ri_buf[i].i_addr, + item->ri_buf[i].i_len); +return -EFSCORRUPTED ; + } i++; }
fs/xfs/xfs_attr_list.c +12 -5
@@ -58,9 +58,13 @@ xfs_attr_shortform_list(struct xfs_attr_sf_sort *sbuf, *sbp;struct xfs_attr_shortform *sf;struct xfs_attr_sf_entry *sfe; +struct xfs_mount *mp;int sbsize, nsbuf, count, i;int error =0 ; +ASSERT (context !=NULL ); +ASSERT (dp !=NULL ); + mp = dp->i_mount; sf = (struct xfs_attr_shortform *)dp->i_af.if_u1.if_data;ASSERT (sf !=NULL );if (!sf->hdr.count)
@@ -82,8 +86,9 @@ xfs_attr_shortform_list( (dp->i_af.if_bytes + sf->hdr.count * 16) < context->bufsize)) {for (i =0 , sfe = &sf->list[0 ]; i < sf->hdr.count; i++) {if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sfe->nameval, - sfe->namelen))) + !xfs_attr_namecheck (mp, sfe->nameval, + sfe->namelen, + sfe->flags)))return -EFSCORRUPTED ; context->put_listent(context, sfe->flags,
@@ -174,8 +179,9 @@ xfs_attr_shortform_list( cursor->offset =0 ; }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sbp->name, - sbp->namelen))) { + !xfs_attr_namecheck (mp, sbp->name, + sbp->namelen, + sbp->flags))) { error = -EFSCORRUPTED ; goto out; }
@@ -465,7 +471,8 @@ xfs_attr3_leaf_list_int( }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (name, namelen))) + !xfs_attr_namecheck (mp, name, namelen, + entry->flags)))return -EFSCORRUPTED ; context->put_listent (context, entry->flags, name, namelen, valuelen);
PATCH v11 7/23 xfs: extend transaction reservations for parent attributes
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.
--- fs/xfs/libxfs/xfs_trans_resv.c | 326 ++++++++++++++++++++++++++++++++++------ 1 file changed, 274 insertions(+), 52 deletions(-)
fs/xfs/libxfs/xfs_trans_resv.c +274 -52
@@ -19,6 +19,9 @@#include "xfs_trans.h" #include "xfs_qm.h" #include "xfs_trans_space.h" +#include "xfs_attr_item.h" +#include "xfs_log.h" +#include "xfs_da_format.h" #define _ALLOC true#define _FREE false
@@ -420,29 +423,110 @@ xfs_calc_itruncate_reservation_minlogsize(return xfs_calc_itruncate_reservation (mp, true); } +static inline unsigned int xfs_calc_pptr_link_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ); +} +static inline unsigned int xfs_calc_pptr_unlink_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ); +} +static inline unsigned int xfs_calc_pptr_replace_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ); +} +/* * In renaming a files we can modify: * the five inodes involved: 5 * inode size * the two directory btrees: 2 * (max depth + v2) * dir block size * the two directory bmap btrees: 2 * max depth * block size * And the bmap_finish transaction can free dir and bmap blocks (two sets -* of bmap blocks) giving: +* of bmap blocks) giving (t2): * the agf for the ags in which the blocks live: 3 * sector size * the agfl for the ags in which the blocks live: 3 * sector size * the superblock for the free block count: sector size * the allocation btrees: 3 exts * 2 trees * (2 * max depth - 1) * block size +* If parent pointers are enabled (t3), then each transaction in the chain +* must be capable of setting or removing the extended attribute +* containing the parent information. It must also be able to handle +* the three xattr intent items that track the progress of the parent +* pointer update. */ STATIC uintxfs_calc_rename_reservation (struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max ((xfs_calc_inode_res (mp,5 ) + -xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_inode_res (mp,5 ) + +xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), +XFS_FSB_TO_B (mp,1 )); + + t2 =xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { +unsigned int rename_overhead, exchange_overhead; + + t3 =max (resp->tr_attrsetm.tr_logres, + resp->tr_attrrm.tr_logres); + +/* +* For a standard rename, the three xattr intent log items +* are (1) replacing the pptr for the source file; (2) +* removing the pptr on the dest file; and (3) adding a +* pptr for the whiteout file in the src dir. +* +* For an RENAME_EXCHANGE, there are two xattr intent +* items to replace the pptr for both src and dest +* files. Link counts don't change and there is no +* whiteout. +* +* In the worst case we can end up relogging all log +* intent items to allow the log tail to move ahead, so +* they become overhead added to each transaction in a +* processing chain. +*/ + rename_overhead =xfs_calc_pptr_replace_overhead () + +xfs_calc_pptr_unlink_overhead () + +xfs_calc_pptr_link_overhead (); + exchange_overhead =2 *xfs_calc_pptr_replace_overhead (); + + overhead +=max (rename_overhead, exchange_overhead); + } + +return overhead +max3 (t1, t2, t3); +} + +static inline unsigned int +xfs_rename_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +/* One for the rename, one more for freeing blocks */ +unsigned int ret =XFS_RENAME_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to remove or add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret +=max (resp->tr_attrsetm.tr_logcount, + resp->tr_attrrm.tr_logcount); + +return ret; }/*
@@ -459,6 +543,23 @@ xfs_calc_iunlink_remove_reservation(2 *M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_link_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_LINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * For creating a link to an inode: * the parent directory inode: inode size
@@ -475,14 +576,23 @@ STATIC uintxfs_calc_link_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_remove_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_remove_reservation (mp); + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -497,6 +607,23 @@ xfs_calc_iunlink_add_reservation(xfs_mount_t *mp)M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_remove_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_REMOVE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrrm.tr_logcount; + +return ret; +} +/* * For removing a directory entry we can modify: * the parent directory inode: inode size
@@ -513,14 +640,24 @@ STATIC uintxfs_calc_remove_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_add_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_add_reservation (mp); + + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrrm.tr_logres; + overhead +=xfs_calc_pptr_unlink_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -569,12 +706,40 @@ xfs_calc_icreate_resv_alloc(xfs_calc_finobt_res (mp); } +static inline unsigned int +xfs_icreate_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_CREATE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} + STATIC uint -xfs_calc_icreate_reservation (xfs_mount_t *mp) +xfs_calc_icreate_reservation ( +struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max (xfs_calc_icreate_resv_alloc (mp), -xfs_calc_create_resv_modify (mp)); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_icreate_resv_alloc (mp); + t2 =xfs_calc_create_resv_modify (mp); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }STATIC uint
@@ -587,6 +752,23 @@ xfs_calc_create_tmpfile_reservation(return res +xfs_calc_iunlink_add_reservation (mp); } +static inline unsigned int +xfs_mkdir_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_MKDIR_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * Making a new directory is the same as creating a new file. */
@@ -597,6 +779,22 @@ xfs_calc_mkdir_reservation(return xfs_calc_icreate_reservation (mp); } +static inline unsigned int +xfs_symlink_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_SYMLINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +}/* * Making a new symplink is the same as creating a new file, but
@@ -909,6 +1107,52 @@ xfs_calc_sb_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize); } +/* +* Namespace reservations. +* +* These get tricky when parent pointers are enabled as we have attribute +* modifications occurring from within these transactions. Rather than confuse +* each of these reservation calculations with the conditional attribute +* reservations, add them here in a clear and concise manner. This requires that +* the attribute reservations have already been calculated. +* +* Note that we only include the static attribute reservation here; the runtime +* reservation will have to be modified by the size of the attributes being +* added/removed/modified. See the comments on the attribute reservation +* calculations for more details. +*/ +STATIC void +xfs_calc_namespace_reservations ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +ASSERT (resp->tr_attrsetm.tr_logres >0 ); + + resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); + resp->tr_rename.tr_logcount =xfs_rename_log_count (mp, resp); + resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); + resp->tr_link.tr_logcount =xfs_link_log_count (mp, resp); + resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); + resp->tr_remove.tr_logcount =xfs_remove_log_count (mp, resp); + resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); + resp->tr_symlink.tr_logcount =xfs_symlink_log_count (mp, resp); + resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); + resp->tr_create.tr_logcount =xfs_icreate_log_count (mp, resp); + resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); + resp->tr_mkdir.tr_logcount =xfs_mkdir_log_count (mp, resp); + resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +} + void xfs_trans_resv_calc( struct xfs_mount *mp,
@@ -928,35 +1172,11 @@ xfs_trans_resv_calc( resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); - resp->tr_rename.tr_logcount =XFS_RENAME_LOG_COUNT ; - resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); - resp->tr_link.tr_logcount =XFS_LINK_LOG_COUNT ; - resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); - resp->tr_remove.tr_logcount =XFS_REMOVE_LOG_COUNT ; - resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); - resp->tr_symlink.tr_logcount =XFS_SYMLINK_LOG_COUNT ; - resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); - resp->tr_create.tr_logcount =XFS_CREATE_LOG_COUNT ; - resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_create_tmpfile.tr_logres =xfs_calc_create_tmpfile_reservation (mp); resp->tr_create_tmpfile.tr_logcount =XFS_CREATE_TMPFILE_LOG_COUNT ; resp->tr_create_tmpfile.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); - resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; - resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_ifree.tr_logres =xfs_calc_ifree_reservation (mp); resp->tr_ifree.tr_logcount =XFS_INACTIVE_LOG_COUNT ; resp->tr_ifree.tr_logflags |=XFS_TRANS_PERM_LOG_RES ;
@@ -986,6 +1206,8 @@ xfs_trans_resv_calc( resp->tr_qm_dqalloc.tr_logcount =XFS_WRITE_LOG_COUNT ; resp->tr_qm_dqalloc.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +xfs_calc_namespace_reservations (mp, resp); +/* * The following transactions are logged in logical format with * a default log count.
PATCH v11 8/23 xfs: parent pointer attribute creation
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add parent pointer attribute during xfs_create, and subroutines to initialize attributes
--- fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_attr.h | 4 +- fs/xfs/libxfs/xfs_parent.c | 115 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 44 ++++++++++++++++- fs/xfs/xfs_inode.c | 57 +++++++++++++++++++--- fs/xfs/xfs_inode.h | 6 ++ fs/xfs/xfs_super.c | 10 ++++ fs/xfs/xfs_xattr.c | 4 +- fs/xfs/xfs_xattr.h | 2 + 9 files changed, 233 insertions(+), 13 deletions(-)
fs/xfs/libxfs/xfs_attr.c +2 -2
@@ -895,7 +895,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set or remove) */
@@ -913,7 +913,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +3 -1
@@ -549,6 +549,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
@@ -557,7 +558,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/* * Check to see if the attr should be upgraded from non-existent or shortform to * single-leaf-block attribute list.
fs/xfs/libxfs/xfs_parent.c +115 -0
124 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +43 -1
47 lines not on this page · raw message
fs/xfs/xfs_inode.c +50 -7
131 lines not on this page · raw message
fs/xfs/xfs_inode.h +6 -0
@@ -153,6 +153,12 @@ static inline struct inode *VFS_I(struct xfs_inode *ip)return &ip->i_vnode; } +/* convert from const xfs inode to vfs inode */ +static inline const struct inode *VFS_IC (const struct xfs_inode *ip) +{ +return &ip->i_vnode; +} +/* * For regular files we only update the on-disk filesize when actually * writing data back to disk. Until then only the copy in the VFS inode
fs/xfs/xfs_super.c +10 -0
@@ -42,6 +42,7 @@#include "xfs_xattr.h" #include "xfs_iunlink_item.h" #include "xfs_dahash_test.h" +#include "xfs_parent.h" #include <linux/magic.h> #include <linux/fs_context.h>
@@ -2125,8 +2126,16 @@ xfs_init_caches(void)if (!xfs_iunlink_cache) goto out_destroy_attri_cache; + xfs_parent_intent_cache =kmem_cache_create ("xfs_parent_intent" , +sizeof (struct xfs_parent_defer ), +0 ,0 ,NULL ); +if (!xfs_parent_intent_cache) + goto out_destroy_iul_cache; +return 0 ; + out_destroy_iul_cache: +kmem_cache_destroy (xfs_iunlink_cache); out_destroy_attri_cache:kmem_cache_destroy (xfs_attri_cache); out_destroy_attrd_cache:
@@ -2181,6 +2190,7 @@ xfs_destroy_caches(void)* destroy caches. */ rcu_barrier (); +kmem_cache_destroy (xfs_parent_intent_cache);kmem_cache_destroy (xfs_iunlink_cache);kmem_cache_destroy (xfs_attri_cache);kmem_cache_destroy (xfs_attrd_cache);
fs/xfs/xfs_xattr.c +2 -2
@@ -27,7 +27,7 @@* they must release the permission by calling xlog_drop_incompat_feat * when they're done. */ -static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {
@@ -61,7 +61,7 @@ xfs_attr_grab_log_assist(return error; } -static inline void +void xfs_attr_rele_log_assist (struct xfs_mount *mp) {
fs/xfs/xfs_xattr.h +2 -0
@@ -7,6 +7,8 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp); +void xfs_attr_rele_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *xfs_xattr_handlers[];
PATCH v11 9/23 xfs: add parent attributes to link
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_link to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.h | 2 - fs/xfs/xfs_inode.c | 56 +++++++++++++++++++++++++++++++++++---- 2 files changed, 50 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -84,8 +84,6 @@ (2 * (mp)->m_alloc_maxlevels)#define XFS_GROWFSRT_SPACE_RES (mp,b) \ ((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK)) -#define XFS_LINK_SPACE_RES (mp,nl) \ - XFS_DIRENTER_SPACE_RES(mp,nl)#define XFS_MKDIR_SPACE_RES (mp,nl) \ (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_QM_DQALLOC_SPACE_RES (mp) \
fs/xfs/xfs_inode.c +50 -6
95 lines not on this page · raw message
PATCH v11 10/23 xfs: add parent attributes to symlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_symlink to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.h | 2 -- fs/xfs/xfs_symlink.c | 49 +++++++++++++++++++++++++++++++++------ 2 files changed, 42 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -95,8 +95,6 @@ XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl)) -#define XFS_SYMLINK_SPACE_RES (mp,nl,b) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl) + (b))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
fs/xfs/xfs_symlink.c +42 -7
111 lines not on this page · raw message
PATCH v11 11/23 xfs: remove parent pointers in unlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the parent pointer attribute during unlink
--- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr.h | 1 + fs/xfs/libxfs/xfs_parent.c | 22 ++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 4 ++++ fs/xfs/libxfs/xfs_trans_space.h | 2 -- fs/xfs/xfs_inode.c | 39 +++++++++++++++++++++++++++++++++------ 6 files changed, 61 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
@@ -959,7 +959,7 @@ xfs_attr_defer_replace( }/* Removes an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_remove (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +1 -0
@@ -550,6 +550,7 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args); +int xfs_attr_defer_remove (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
fs/xfs/libxfs/xfs_parent.c +22 -0
@@ -182,6 +182,28 @@ xfs_parent_add(return xfs_attr_defer_add (args); } +/* Remove a parent pointer to reflect a dirent removal. */ +int +xfs_parent_remove ( +struct xfs_trans *tp, +struct xfs_parent_defer *parent, +struct xfs_inode *dp, +const struct xfs_name *parent_name, +struct xfs_inode *child) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, dp, parent_name, child); + args->hashval =xfs_parent_hashname (dp, parent); + + args->trans = tp; + args->dp = child; + +xfs_init_parent_davalue (&parent->args, parent_name); + +return xfs_attr_defer_remove (args); +} +/* Cancel a parent pointer operation. */ void __xfs_parent_cancel(
fs/xfs/libxfs/xfs_parent.h +4 -0
@@ -41,6 +41,10 @@ xfs_parent_start(int xfs_parent_add (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp,const struct xfs_name *parent_name,struct xfs_inode *child); +int xfs_parent_remove (struct xfs_trans *tp,struct xfs_parent_defer *parent, +struct xfs_inode *dp,const struct xfs_name *parent_name, +struct xfs_inode *child); +void __xfs_parent_cancel (struct xfs_mount *mp,struct xfs_parent_defer *parent); static inline void
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_REMOVE_SPACE_RES (mp) \ - XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \
fs/xfs/xfs_inode.c +33 -6
@@ -2472,6 +2472,19 @@ xfs_iunpin_wait(__xfs_iunpin_wait (ip); } +static unsigned int +xfs_remove_space_res ( +struct xfs_mount *mp, +unsigned int namelen) +{ +unsigned int ret =XFS_DIRREMOVE_SPACE_RES (mp); + +if (xfs_has_parent (mp)) + ret +=xfs_parent_calc_space_res (mp, namelen); + +return ret; +} +/* * Removing an inode from the namespace involves removing the directory entry * and dropping the link count on the inode. Removing the directory entry can
@@ -2501,16 +2514,17 @@ xfs_iunpin_wait(*/ int xfs_remove ( -xfs_inode_t *dp, +struct xfs_inode *dp,struct xfs_name *name, -xfs_inode_t *ip) +struct xfs_inode *ip) { -xfs_mount_t *mp = dp->i_mount; -xfs_trans_t *tp =NULL ; +struct xfs_mount *mp = dp->i_mount; +struct xfs_trans *tp =NULL ;int is_dir =S_ISDIR (VFS_I (ip)->i_mode);int dontcare;int error =0 ;uint resblks; +struct xfs_parent_defer *parent =NULL ;trace_xfs_remove (dp, name);
@@ -2525,6 +2539,10 @@ xfs_remove(if (error) goto std_return; + error =xfs_parent_start (mp, &parent); +if (error) + goto std_return; +/* * We try to get the real space reservation first, allowing for * directory btree deletion(s) implying possible bmap insert(s). If we
@@ -2536,12 +2554,12 @@ xfs_remove(* the directory code can handle a reservationless update and we don't * want to prevent a user from trying to free space by deleting things. */ - resblks =XFS_REMOVE_SPACE_RES (mp); + resblks =xfs_remove_space_res (mp, name->len); error =xfs_trans_alloc_dir (dp, &M_RES (mp)->tr_remove, ip, &resblks, &tp, &dontcare);if (error) {ASSERT (error != -ENOSPC ); - goto std_return; + goto out_parent; }/*
@@ -2601,6 +2619,12 @@ xfs_remove( goto out_trans_cancel; } +if (parent) { + error =xfs_parent_remove (tp, parent, dp, name, ip); +if (error) + goto out_trans_cancel; + } +/* * If this is a synchronous mount, make sure that the * remove transaction goes to disk before returning to
@@ -2618,6 +2642,7 @@ xfs_remove(xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); +xfs_parent_finish (mp, parent);return 0 ; out_trans_cancel:
@@ -2625,6 +2650,8 @@ xfs_remove( out_unlock:xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); + out_parent: +xfs_parent_finish (mp, parent); std_return:return error; }
PATCH v11 12/23 xfs: Indent xfs_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Indent variables and parameters in xfs_rename in preparation for parent pointer modifications. White space only, no functional changes. This will make reviewing new code easier on reviewers.
--- fs/xfs/xfs_inode.c | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-)
fs/xfs/xfs_inode.c +20 -19
@@ -2896,26 +2896,27 @@ xfs_rename_alloc_whiteout(*/ int xfs_rename ( -struct mnt_idmap *idmap, -struct xfs_inode *src_dp, -struct xfs_name *src_name, -struct xfs_inode *src_ip, -struct xfs_inode *target_dp, -struct xfs_name *target_name, -struct xfs_inode *target_ip, -unsigned int flags) +struct mnt_idmap *idmap, +struct xfs_inode *src_dp, +struct xfs_name *src_name, +struct xfs_inode *src_ip, +struct xfs_inode *target_dp, +struct xfs_name *target_name, +struct xfs_inode *target_ip, +unsigned int flags) { -struct xfs_mount *mp = src_dp->i_mount; -struct xfs_trans *tp; -struct xfs_inode *wip =NULL ;/* whiteout inode */ -struct xfs_inode *inodes[__XFS_SORT_INODES]; -int i; -int num_inodes = __XFS_SORT_INODES; -bool new_parent = (src_dp != target_dp); -bool src_is_directory =S_ISDIR (VFS_I (src_ip)->i_mode); -int spaceres; -bool retried = false; -int error, nospace_error =0 ; +struct xfs_mount *mp = src_dp->i_mount; +struct xfs_trans *tp; +struct xfs_inode *wip =NULL ;/* whiteout inode */ +struct xfs_inode *inodes[__XFS_SORT_INODES]; +int i; +int num_inodes = __XFS_SORT_INODES; +bool new_parent = (src_dp != target_dp); +bool src_is_directory = +S_ISDIR (VFS_I (src_ip)->i_mode); +int spaceres; +bool retried = false; +int error, nospace_error =0 ;trace_xfs_rename (src_dp, target_dp, src_name, target_name);
PATCH v11 13/23 xfs: Add parent pointers to rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer.
--- fs/xfs/libxfs/xfs_attr.c | 2 - fs/xfs/libxfs/xfs_attr.h | 1 fs/xfs/libxfs/xfs_parent.c | 58 ++++++++++++++++++++-- fs/xfs/libxfs/xfs_parent.h | 23 ++++++++- fs/xfs/libxfs/xfs_trans_space.h | 2 - fs/xfs/xfs_inode.c | 103 +++++++++++++++++++++++++++++++++++++-- 6 files changed, 174 insertions(+), 15 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
@@ -936,7 +936,7 @@ xfs_attr_defer_add( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_replace (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +1 -0
@@ -551,6 +551,7 @@ int xfs_attr_get_ilocked(struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_defer_remove (struct xfs_da_args *args); +int xfs_attr_defer_replace (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
fs/xfs/libxfs/xfs_parent.c +53 -5
94 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +21 -2
@@ -21,10 +21,13 @@ extern struct kmem_cache *xfs_parent_intent_cache;*/ struct xfs_parent_defer {struct xfs_parent_name_rec rec; +struct xfs_parent_name_rec new_rec;struct xfs_da_args args; +bool have_log; }; -int __xfs_parent_init (struct xfs_mount *mp,struct xfs_parent_defer **parentp); +int __xfs_parent_init (struct xfs_mount *mp,bool grab_log, +struct xfs_parent_defer **parentp);static inline int xfs_parent_start(
@@ -34,7 +37,19 @@ xfs_parent_start( *pp =NULL ;if (xfs_has_parent (mp)) -return __xfs_parent_init (mp, pp); +return __xfs_parent_init (mp, true, pp); +return 0 ; +} + +static inline int +xfs_parent_start_locked ( +struct xfs_mount *mp, +struct xfs_parent_defer **pp) +{ + *pp =NULL ; + +if (xfs_has_parent (mp)) +return __xfs_parent_init (mp, false, pp);return 0 ; }
@@ -44,6 +59,10 @@ int xfs_parent_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,int xfs_parent_remove (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp,const struct xfs_name *parent_name,struct xfs_inode *child); +int xfs_parent_replace (struct xfs_trans *tp,struct xfs_parent_defer *parent, +struct xfs_inode *old_dp,const struct xfs_name *old_name, +struct xfs_inode *new_dp,const struct xfs_name *new_name, +struct xfs_inode *child);void __xfs_parent_cancel (struct xfs_mount *mp,struct xfs_parent_defer *parent);
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_RENAME_SPACE_RES (mp,nl) \ - (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
fs/xfs/xfs_inode.c +98 -5
158 lines not on this page · raw message
PATCH v11 14/23 xfs: Add parent pointers to xfs_cross_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Cross renames are handled separately from standard renames, and need different handling to update the parent attributes correctly.
--- fs/xfs/xfs_inode.c | 44 ++++++++++++++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 14 deletions(-)
fs/xfs/xfs_inode.c +30 -14
@@ -2743,19 +2743,22 @@ xfs_finish_rename(*/ STATIC intxfs_cross_rename ( -struct xfs_trans *tp, -struct xfs_inode *dp1, -struct xfs_name *name1, -struct xfs_inode *ip1, -struct xfs_inode *dp2, -struct xfs_name *name2, -struct xfs_inode *ip2, -int spaceres) +struct xfs_trans *tp, +struct xfs_inode *dp1, +struct xfs_name *name1, +struct xfs_inode *ip1, +struct xfs_parent_defer *ip1_pptr, +struct xfs_inode *dp2, +struct xfs_name *name2, +struct xfs_inode *ip2, +struct xfs_parent_defer *ip2_pptr, +int spaceres) { -int error =0 ; -int ip1_flags =0 ; -int ip2_flags =0 ; -int dp2_flags =0 ; +struct xfs_mount *mp = dp1->i_mount; +int error =0 ; +int ip1_flags =0 ; +int ip2_flags =0 ; +int dp2_flags =0 ;/* Swap inode number for dirent in first parent */ error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres);
@@ -2824,6 +2827,18 @@ xfs_cross_rename( } } +if (xfs_has_parent (mp)) { + error =xfs_parent_replace (tp, ip1_pptr, dp1, name1, dp2, + name2, ip1); +if (error) + goto out_trans_abort; + + error =xfs_parent_replace (tp, ip2_pptr, dp2, name2, dp1, + name1, ip2); +if (error) + goto out_trans_abort; + } +if (ip1_flags) {xfs_trans_ichgtime (tp, ip1, ip1_flags);xfs_trans_log_inode (tp, ip1,XFS_ILOG_CORE );
@@ -2838,6 +2853,7 @@ xfs_cross_rename( }xfs_trans_ichgtime (tp, dp1,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, dp1,XFS_ILOG_CORE ); +return xfs_finish_rename (tp); out_trans_abort:
@@ -3052,8 +3068,8 @@ xfs_rename(/* RENAME_EXCHANGE is unique from here on. */ if (flags &RENAME_EXCHANGE ) { error =xfs_cross_rename (tp, src_dp, src_name, src_ip, - target_dp, target_name, target_ip, - spaceres); + src_ip_pptr, target_dp, target_name, target_ip, + tgt_ip_pptr, spaceres); goto out_unlock; }
PATCH v11 15/23 xfs: Add helper function xfs_attr_list_context_init
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch adds a helper function xfs_attr_list_context_init used by xfs_attr_list. This function initializes the xfs_attr_list_context structure passed to xfs_attr_list_int. We will need this later to call xfs_attr_list_int_ilocked when the node is already locked.
--- fs/xfs/xfs_file.c | 1 + fs/xfs/xfs_ioctl.c | 54 +++++++++++++++++++++++++++++++++++++--------------- fs/xfs/xfs_ioctl.h | 2 ++ 3 files changed, 41 insertions(+), 16 deletions(-)
fs/xfs/xfs_file.c +1 -0
@@ -17,6 +17,7 @@#include "xfs_bmap_util.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" #include "xfs_ioctl.h" #include "xfs_trace.h" #include "xfs_log.h"
fs/xfs/xfs_ioctl.c +38 -16
72 lines not on this page · raw message
fs/xfs/xfs_ioctl.h +2 -0
@@ -35,6 +35,8 @@ int xfs_ioc_attrmulti_one(struct file *parfilp, struct inode *inode,int xfs_ioc_attr_list (struct xfs_inode *dp,void __user *ubuf,size_t bufsize,int flags,struct xfs_attrlist_cursor __user *ucursor); +int xfs_ioc_attr_list_context_init (struct xfs_inode *dp,char *buffer, +int bufsize,int flags,struct xfs_attr_list_context *context);extern struct dentry * xfs_handle_to_dentry(
PATCH v11 16/23 xfs: Filter XFS_ATTR_PARENT for getfattr
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Parent pointers returned to the get_fattr tool cause errors since the tool cannot parse parent pointers. Fix this by filtering parent parent pointers from xfs_xattr_put_listent.
--- fs/xfs/xfs_xattr.c | 3 +++ 1 file changed, 3 insertions(+)
fs/xfs/xfs_xattr.c +3 -0
@@ -239,6 +239,9 @@ xfs_xattr_put_listent(ASSERT (context->count >=0 ); +if (flags &XFS_ATTR_PARENT ) +return ; +if (flags &XFS_ATTR_ROOT ) {#ifdef CONFIG_XFS_POSIX_ACL if (namelen == SGI_ACL_FILE_SIZE &&
PATCH v11 17/23 xfs: pass the attr value to put_listent when possible
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Pass the attr value to put_listent when we have local xattrs or shortform xattrs.
--- fs/xfs/libxfs/xfs_attr.h | 5 +++-- fs/xfs/libxfs/xfs_attr_sf.h | 1 + fs/xfs/scrub/attr.c | 8 ++++++++ fs/xfs/xfs_attr_list.c | 8 +++++++- fs/xfs/xfs_ioctl.c | 1 + fs/xfs/xfs_xattr.c | 1 + 6 files changed, 21 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_attr.h +3 -2
@@ -47,8 +47,9 @@ struct xfs_attrlist_cursor_kern {/* void; state communicated via *context */ -typedef void (*put_listent_func_t )(struct xfs_attr_list_context *,int , -unsigned char *,int ,int ); +typedef void (*put_listent_func_t )(struct xfs_attr_list_context *context, +int flags,unsigned char *name,int namelen,void *value, +int valuelen);struct xfs_attr_list_context {struct xfs_trans *tp;
fs/xfs/libxfs/xfs_attr_sf.h +1 -0
@@ -24,6 +24,7 @@ typedef struct xfs_attr_sf_sort {uint8_t flags;/* flags bits (see xfs_attr_leaf.h) */ xfs_dahash_t hash;/* this entry's hash value */ unsigned char *name;/* name value, pointer into buffer */ +void *value; } xfs_attr_sf_sort_t;#define XFS_ATTR_SF_ENTSIZE_MAX /* max space for name&value */ \
fs/xfs/scrub/attr.c +8 -0
@@ -109,6 +109,7 @@ xchk_xattr_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { struct xchk_xattr *sx;
@@ -134,6 +135,13 @@ xchk_xattr_listent(return ; } +/* +* Shortform and local attrs don't require external lookups to retrieve +* the value, so there's nothing else to check here. +*/ +if (value) +return ; +/* * Try to allocate enough memory to extrat the attr value. If that * doesn't work, we overload the seen_enough variable to convey
fs/xfs/xfs_attr_list.c +7 -1
@@ -94,6 +94,7 @@ xfs_attr_shortform_list( sfe->flags, sfe->nameval, (int )sfe->namelen, + &sfe->nameval[sfe->namelen], (int)sfe->valuelen);/* * Either search callback finished early or
@@ -139,6 +140,7 @@ xfs_attr_shortform_list( sbp->name = sfe->nameval; sbp->namelen = sfe->namelen;/* These are bytes, and both on-disk, don't endian-flip */ + sbp->value = &sfe->nameval[sfe->namelen], sbp->valuelen = sfe->valuelen; sbp->flags = sfe->flags; sfe =xfs_attr_sf_nextentry (sfe);
@@ -189,6 +191,7 @@ xfs_attr_shortform_list( sbp->flags, sbp->name, sbp->namelen, + sbp->value, sbp->valuelen);if (context->seen_enough)break ;
@@ -443,6 +446,7 @@ xfs_attr3_leaf_list_int(*/ for (; i < ichdr.count; entry++, i++) {char *name; +void *value;int namelen, valuelen;if (be32_to_cpu (entry->hashval) != cursor->hashval) {
@@ -460,6 +464,7 @@ xfs_attr3_leaf_list_int( name_loc =xfs_attr3_leaf_name_local (leaf, i); name = name_loc->nameval; namelen = name_loc->namelen; + value = &name_loc->nameval[name_loc->namelen]; valuelen =be16_to_cpu (name_loc->valuelen); }else {xfs_attr_leaf_name_remote_t *name_rmt;
@@ -467,6 +472,7 @@ xfs_attr3_leaf_list_int( name_rmt =xfs_attr3_leaf_name_remote (leaf, i); name = name_rmt->name; namelen = name_rmt->namelen; + value =NULL ; valuelen =be32_to_cpu (name_rmt->valuelen); }
@@ -475,7 +481,7 @@ xfs_attr3_leaf_list_int( entry->flags)))return -EFSCORRUPTED ; context->put_listent (context, entry->flags, - name, namelen, valuelen); + name, namelen, value, valuelen);if (context->seen_enough)break ; cursor->offset++;
fs/xfs/xfs_ioctl.c +1 -0
@@ -307,6 +307,7 @@ xfs_ioc_attr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { struct xfs_attrlist *alist = context->buffer;
fs/xfs/xfs_xattr.c +1 -0
@@ -232,6 +232,7 @@ xfs_xattr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { char *prefix;
PATCH v11 18/23 xfs: add a libxfs header file for staging new ioctls
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new xfs_fs_staging.h header where we can land experimental ioctls without committing them to any stable interfaces anywhere.
--- fs/xfs/libxfs/xfs_fs_staging.h | 18 ++++++++++++++++++ fs/xfs/xfs_linux.h | 1 + 2 files changed, 19 insertions(+) create mode 100644 fs/xfs/libxfs/xfs_fs_staging.h
fs/xfs/libxfs/xfs_fs_staging.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: LGPL-2.1 */ +/* +* Copyright (C) 2020-2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_FS_STAGING_H__ +#define __XFS_FS_STAGING_H__ + +/* +* Experimental system calls, ioctls and data structures supporting them. +* Nothing in here should be considered part of a stable interface of any kind. +* +* If you add an ioctl here, please leave a comment in xfs_fs.h marking it +* reserved. If you promote anything out of this file, please leave a comment +* explaining where it went. +*/ + +#endif /* __XFS_FS_STAGING_H__ */
fs/xfs/xfs_linux.h +1 -0
@@ -72,6 +72,7 @@ typedef __u32 xfs_nlink_t;#include <asm/unaligned.h> #include "xfs_fs.h" +#include "xfs_fs_staging.h" #include "xfs_stats.h" #include "xfs_sysctl.h" #include "xfs_iops.h"
PATCH v11 19/23 xfs: Add parent pointer ioctl
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch adds a new file ioctl to retrieve the parent pointer of a given inode
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_fs.h | 1 fs/xfs/libxfs/xfs_fs_staging.h | 66 +++++++++++++++++ fs/xfs/libxfs/xfs_parent.c | 40 ++++++++++ fs/xfs/libxfs/xfs_parent.h | 23 ++++++ fs/xfs/xfs_ioctl.c | 134 ++++++++++++++++++++++++++++++++++ fs/xfs/xfs_ondisk.h | 4 + fs/xfs/xfs_parent_utils.c | 157 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 20 +++++ fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 73 +++++++++++++++++++ 11 files changed, 519 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h
fs/xfs/Makefile +1 -0
@@ -87,6 +87,7 @@ xfs-y += xfs_aops.o \ xfs_mount.o \ xfs_mru_cache.o \ xfs_pwork.o \ + xfs_parent_utils.o \ xfs_reflink.o \ xfs_stats.o \ xfs_super.o \
fs/xfs/libxfs/xfs_fs.h +1 -0
@@ -796,6 +796,7 @@ struct xfs_scrub_metadata {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +/* XFS_IOC_GETPARENTS ---- staging 62 */ /* * ioctl commands that replace IRIX syssgi()'s
fs/xfs/libxfs/xfs_fs_staging.h +66 -0
70 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.c +40 -0
@@ -275,3 +275,43 @@ xfs_parent_calc_space_res(return XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ) +XFS_NEXTENTADD_SPACE_RES (mp, namelen,XFS_ATTR_FORK ); } + +/* Convert an ondisk parent pointer to the incore format. */ +void +xfs_parent_irec_from_disk ( +struct xfs_parent_name_irec *irec, +const struct xfs_parent_name_rec *rec, +const void *value, +unsigned int valuelen) +{ + irec->p_ino =be64_to_cpu (rec->p_ino); + irec->p_gen =be32_to_cpu (rec->p_gen); + irec->p_namehash =be32_to_cpu (rec->p_namehash); + irec->p_namelen = valuelen; +memcpy (irec->p_name, value, valuelen); +} + +/* Convert an incore parent pointer to the ondisk attr name format. */ +void +xfs_parent_irec_to_disk ( +struct xfs_parent_name_rec *rec, +const struct xfs_parent_name_irec *irec) +{ + rec->p_ino =cpu_to_be64 (irec->p_ino); + rec->p_gen =cpu_to_be32 (irec->p_gen); + rec->p_namehash =cpu_to_be32 (irec->p_namehash); +} + +/* Compute p_namehash for the this parent pointer. */ +void +xfs_parent_irec_hashname ( +struct xfs_mount *mp, +struct xfs_parent_name_irec *irec) +{ +struct xfs_name dname = { + .name = irec->p_name, + .len = irec->p_namelen, + }; + + irec->p_namehash =xfs_dir2_hashname (mp, &dname); +}
fs/xfs/libxfs/xfs_parent.h +23 -0
@@ -78,4 +78,27 @@ xfs_parent_finish(unsigned int xfs_parent_calc_space_res (struct xfs_mount *mp,unsigned int namelen); +/* +* Incore version of a parent pointer, also contains dirent name so callers +* can pass/obtain all the parent pointer information in a single structure +*/ +struct xfs_parent_name_irec { +/* Parent pointer attribute name fields */ +xfs_ino_t p_ino; +uint32_t p_gen; +xfs_dahash_t p_namehash; + +/* Parent pointer attribute value fields */ +uint8_t p_namelen; +unsigned char p_name[MAXNAMELEN ]; +}; + +void xfs_parent_irec_from_disk (struct xfs_parent_name_irec *irec, +const struct xfs_parent_name_rec *rec,const void *value, +unsigned int valuelen); +void xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec, +const struct xfs_parent_name_irec *irec); +void xfs_parent_irec_hashname (struct xfs_mount *mp, +struct xfs_parent_name_irec *irec); +#endif /* __XFS_PARENT_H__ */
fs/xfs/xfs_ioctl.c +133 -1
152 lines not on this page · raw message
fs/xfs/xfs_ondisk.h +4 -0
@@ -150,6 +150,10 @@ xfs_check_ondisk_structs(void) XFS_CHECK_OFFSET(struct xfs_efi_log_format_32, efi_extents, 16); XFS_CHECK_OFFSET(struct xfs_efi_log_format_64, efi_extents, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents_rec, 24); + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents, 96); +/* * The v5 superblock format extended several v4 header structures with * additional data. While new fields are only accessible on v5
fs/xfs/xfs_parent_utils.c new file +157 -0
157 lines not on this page · raw message
fs/xfs/xfs_parent_utils.h new file +20 -0
@@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022-2023 Oracle, Inc. +* All rights reserved. +*/ +#ifndef __XFS_PARENT_UTILS_H__ +#define __XFS_PARENT_UTILS_H__ + +static inline unsigned int +xfs_getparents_arraytop ( +const struct xfs_getparents *ppi, +unsigned int nr) +{ +return sizeof (struct xfs_getparents ) + + (nr *sizeof (ppi->gp_offsets[0 ])); +} + +int xfs_getparent_pointers (struct xfs_inode *ip,struct xfs_getparents *ppi); + +#endif /* __XFS_PARENT_UTILS_H__ */
fs/xfs/xfs_trace.c +1 -0
@@ -36,6 +36,7 @@#include "xfs_error.h" #include <linux/iomap.h> #include "xfs_iomap.h" +#include "xfs_parent.h" /* * We include this last to have the helpers above available for the trace
fs/xfs/xfs_trace.h +73 -0
85 lines not on this page · raw message
PATCH v11 20/23 xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Dave and I were discussing some recent test regressions as a result of me turning on nrext64=1 on realtime filesystems, when we noticed that the minimum log size of a 32M filesystem jumped from 954 blocks to 4287 blocks.
Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size contains the maximum estimated amount of space needed for a local format xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES, which requires units of blocks. This has resulted in an overestimation of the minimum log size over the years.
We should nominally correct this, but there's a backwards compatibility problem -- if we enable it now, the minimum log size will decrease. If a corrected mkfs formats a filesystem with this new smaller log size, a user will encounter mount failures on an uncorrected kernel due to the larger minimum log size computations there.
However, the large extent counters feature is still EXPERIMENTAL, so we can gate the correction on that feature (or any features that get added after that) being enabled. Any filesystem with nrext64 or any of the as-yet-undefined feature bits turned on will be rejected by old uncorrected kernels, so this should be safe even in the upgrade case.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 43 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +43 -0
@@ -16,6 +16,39 @@#include "xfs_bmap_btree.h" #include "xfs_trace.h" +/* +* Decide if the filesystem has the parent pointer feature or any feature +* added after that. +*/ +static inline bool +xfs_has_parent_or_newer_feature ( +struct xfs_mount *mp) +{ +if (!xfs_sb_is_v5 (&mp->m_sb)) +return false; + +if (xfs_sb_has_compat_feature (&mp->m_sb, ~0 )) +return true; + +if (xfs_sb_has_ro_compat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_RO_COMPAT_FINOBT | +XFS_SB_FEAT_RO_COMPAT_RMAPBT | +XFS_SB_FEAT_RO_COMPAT_REFLINK | +XFS_SB_FEAT_RO_COMPAT_INOBTCNT ))) +return true; + +if (xfs_sb_has_incompat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_INCOMPAT_FTYPE | +XFS_SB_FEAT_INCOMPAT_SPINODES | +XFS_SB_FEAT_INCOMPAT_META_UUID | +XFS_SB_FEAT_INCOMPAT_BIGTIME | +XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR | +XFS_SB_FEAT_INCOMPAT_NREXT64 ))) +return true; + +return false; +} +/* * Calculate the maximum length in bytes that would be required for a local * attribute value as large attributes out of line are not logged.
@@ -31,6 +64,16 @@ xfs_log_calc_max_attrsetm_res(MAXNAMELEN -1 ; nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); nblks +=XFS_B_TO_FSB (mp, size); + +/* +* Starting with the parent pointer feature, every new fs feature +* corrects a unit conversion error in the xattr transaction +* reservation code that resulted in oversized minimum log size +* computations. +*/ +if (xfs_has_parent_or_newer_feature (mp)) + size =XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); return M_RES(mp)->tr_attrsetm.tr_logres +
PATCH v11 21/23 xfs: drop compatibility minimum log size computations for reflink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Having established that we can reduce the minimum log size computation for filesystems with parent pointers or any newer feature, we should also drop the compat minlogsize code that we added when we reduced the transaction reservation size for rmap and reflink.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +10 -0
@@ -91,6 +91,16 @@ xfs_log_calc_trans_resv_for_minlogblocks( {unsigned int rmap_maxlevels = mp->m_rmap_maxlevels; +/* +* Starting with the parent pointer feature, every new fs feature +* drops the oversized minimum log size computation introduced by the +* original reflink code. +*/ +if (xfs_has_parent_or_newer_feature (mp)) { +xfs_trans_resv_calc (mp, resv); +return ; + } +/* * In the early days of rmap+reflink, we always set the rmap maxlevels * to 9 even if the AG was small enough that it would never grow to
PATCH v11 22/23 xfs: don't remove the attr fork when parent pointers are enabled
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
When an inode is removed, it may also cause the attribute fork to be removed if it is the last attribute. This transaction gets flushed to the log, but if the system goes down before we could inactivate the symlink, the log recovery tries to inactivate this inode (since it is on the unlinked list) but the verifier trips over the remote value and leaks it.
Hence we ended up with a file in this odd state on a "clean" mount. The "obvious" fix is to prohibit erasure of the attr fork to avoid tripping over the verifiers when pptrs are enabled.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +4 -2
@@ -878,7 +878,8 @@ xfs_attr_sf_removename( totsize -= size;if (totsize ==sizeof (xfs_attr_sf_hdr_t) &&xfs_has_attr2 (mp) && (dp->i_df.if_format !=XFS_DINODE_FMT_BTREE ) && - !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE ))) { + !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE )) && + !xfs_has_parent (mp)) {xfs_attr_fork_remove (dp, args->trans); }else {xfs_idata_realloc (dp, -size,XFS_ATTR_FORK );
@@ -887,7 +888,8 @@ xfs_attr_sf_removename(ASSERT (totsize >sizeof (xfs_attr_sf_hdr_t) || (args->op_flags &XFS_DA_OP_ADDNAME ) || !xfs_has_attr2 (mp) || - dp->i_df.if_format ==XFS_DINODE_FMT_BTREE ); + dp->i_df.if_format ==XFS_DINODE_FMT_BTREE || +xfs_has_parent (mp));xfs_trans_log_inode (args->trans, dp,XFS_ILOG_CORE |XFS_ILOG_ADATA ); }
PATCH v11 23/23 xfs: Add the parent pointer support to the superblock version 5.
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the parent pointer superblock flag so that we can actually mount filesystems with this feature enabled.
--- fs/xfs/libxfs/xfs_format.h | 4 +++- fs/xfs/libxfs/xfs_fs.h | 1 + fs/xfs/libxfs/xfs_sb.c | 4 ++++ fs/xfs/xfs_super.c | 4 ++++ 4 files changed, 12 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +3 -1
@@ -373,13 +373,15 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_BIGTIME (1 << 3)/* large timestamps */ #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)/* needs xfs_repair */ #define XFS_SB_FEAT_INCOMPAT_NREXT64 (1 << 5)/* large extent counters */ +#define XFS_SB_FEAT_INCOMPAT_PARENT (1 << 6)/* parent pointers */ #define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ XFS_SB_FEAT_INCOMPAT_META_UUID| \ XFS_SB_FEAT_INCOMPAT_BIGTIME| \ XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \ - XFS_SB_FEAT_INCOMPAT_NREXT64) + XFS_SB_FEAT_INCOMPAT_NREXT64| \ + XFS_SB_FEAT_INCOMPAT_PARENT)#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
fs/xfs/libxfs/xfs_fs.h +1 -0
@@ -237,6 +237,7 @@ typedef struct xfs_fsop_resblks {#define XFS_FSOP_GEOM_FLAGS_BIGTIME (1 << 21)/* 64-bit nsec timestamps */ #define XFS_FSOP_GEOM_FLAGS_INOBTCNT (1 << 22)/* inobt btree counter */ #define XFS_FSOP_GEOM_FLAGS_NREXT64 (1 << 23)/* large extent counters */ +#define XFS_FSOP_GEOM_FLAGS_PARENT (1 << 24)/* parent pointers */ /* * Minimum and maximum sizes need for growth checks.
fs/xfs/libxfs/xfs_sb.c +4 -0
@@ -173,6 +173,8 @@ xfs_sb_version_to_features( features |=XFS_FEAT_NEEDSREPAIR ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_NREXT64 ) features |=XFS_FEAT_NREXT64 ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_PARENT ) + features |=XFS_FEAT_PARENT ;return features; }
@@ -1190,6 +1192,8 @@ xfs_fs_geometry( geo->flags |=XFS_FSOP_GEOM_FLAGS_BIGTIME ;if (xfs_has_inobtcounts (mp)) geo->flags |=XFS_FSOP_GEOM_FLAGS_INOBTCNT ; +if (xfs_has_parent (mp)) + geo->flags |=XFS_FSOP_GEOM_FLAGS_PARENT ;if (xfs_has_sector (mp)) { geo->flags |=XFS_FSOP_GEOM_FLAGS_SECTOR ; geo->logsectsize = sbp->sb_logsectsize;
fs/xfs/xfs_super.c +4 -0
@@ -1675,6 +1675,10 @@ xfs_fs_fill_super(xfs_warn (mp,"EXPERIMENTAL Large extent counts feature in use. Use at your own risk!" ); +if (xfs_has_parent (mp)) +xfs_alert (mp, +"EXPERIMENTAL parent pointer feature enabled. Use at your own risk!" ); + error =xfs_mountfs (mp);if (error) goto out_filestream_unmount;
v11 cover xfs: online repair of directories
In reply to Darrick J. WongHi all,
With this patchset, we implement online reconstruction of directories by scanning the entire filesystem looking for parent pointer data. This mostly works, except for the part where we need to resync the diroffset field of the parent pointers to match the new directory structure.
Fixing that is left as an open research question, with a few possible solutions:
1. As part of committing the new directory, queue a bunch of parent pointer updates to make those changes.
2. Leave them inconsistent and let the parent pointer repair fix it.
3. Change the ondisk format of parent pointers (and xattrs) so that we can encode the full dirent name in the xattr name.
4. Change the ondisk format of parent pointers to encode a sha256 hash of the dirent name in the xattr name. This will work as long as nobody breaks sha256.
Thoughts? Note that the atomic swapext and block reaping code is NOT ported for this PoC, so we do not commit any repairs.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-repair
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-dir-repair
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_dir2.c | 2 fs/xfs/libxfs/xfs_dir2.h | 2 fs/xfs/scrub/common.c | 15 + fs/xfs/scrub/common.h | 28 + fs/xfs/scrub/dir.c | 9 fs/xfs/scrub/dir_repair.c | 1125 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 16 + fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/tempfile.c | 42 ++ fs/xfs/scrub/tempfile.h | 2 fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 65 +++ 13 files changed, 1307 insertions(+), 3 deletions(-) create mode 100644 fs/xfs/scrub/dir_repair.c
PATCH v11 1/3 xfs: reconstruct directories from parent pointers
From: Darrick J. Wong <[email protected]>
Use the filesystem scanning infrastructure to walk the filesystem looking for parent pointers and child dirents that reference the directory that we're rebuilding.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/common.c | 15 + fs/xfs/scrub/common.h | 28 + fs/xfs/scrub/dir.c | 9 fs/xfs/scrub/dir_repair.c | 964 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 16 + fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/tempfile.c | 42 ++ fs/xfs/scrub/tempfile.h | 2 fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 64 +++ 11 files changed, 1143 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/scrub/dir_repair.c
fs/xfs/Makefile +1 -0
@@ -179,6 +179,7 @@ xfs-$(CONFIG_XFS_QUOTA) += scrub/quota.o ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ + dir_repair.o \ repair.o \ tempfile.o \ xfblob.o \
fs/xfs/scrub/common.c +15 -0
@@ -551,6 +551,21 @@ xchk_ag_init(/* Per-scrubber setup functions */ +void +xchk_trans_cancel ( +struct xfs_scrub *sc) +{ +xfs_trans_cancel (sc->tp); + sc->tp =NULL ; +} + +int +xchk_trans_alloc_empty ( +struct xfs_scrub *sc) +{ +return xfs_trans_alloc_empty (sc->mp, &sc->tp); +} +/* * Grab an empty transaction so that we can re-grab locked buffers if * one of our btrees turns out to be cyclic.
fs/xfs/scrub/common.h +28 -0
@@ -31,6 +31,9 @@ xchk_should_terminate(return false; } +void xchk_trans_cancel (struct xfs_scrub *sc); +int xchk_trans_alloc_empty (struct xfs_scrub *sc); +int xchk_trans_alloc (struct xfs_scrub *sc,uint resblks);bool xchk_process_error (struct xfs_scrub *sc,xfs_agnumber_t agno,xfs_agblock_t bno,int *error);
@@ -159,4 +162,29 @@ void xchk_start_reaping(struct xfs_scrub *sc);void xchk_fshooks_enable (struct xfs_scrub *sc,unsigned int scrub_fshooks); +#ifdef CONFIG_XFS_ONLINE_REPAIR +/* Decide if a repair is required. */ +static inline bool xchk_needs_repair (const struct xfs_scrub_metadata *sm) +{ +return sm->sm_flags & (XFS_SCRUB_OFLAG_CORRUPT | +XFS_SCRUB_OFLAG_XCORRUPT | +XFS_SCRUB_OFLAG_PREEN ); +} + +/* +* "Should we prepare for a repair?" +* +* Return true if the caller permits us to repair metadata and we're not +* setting up for a post-repair evaluation. +*/ +static inline bool xchk_could_repair (const struct xfs_scrub *sc) +{ +return (sc->sm->sm_flags &XFS_SCRUB_IFLAG_REPAIR ) && + !(sc->flags &XREP_ALREADY_FIXED ); +} +#else +# define xchk_needs_repair (sc) (false) +# define xchk_could_repair (sc) (false) +#endif /* CONFIG_XFS_ONLINE_REPAIR */ +#endif /* __XFS_SCRUB_COMMON_H__ */
fs/xfs/scrub/dir.c +9 -0
@@ -19,12 +19,21 @@#include "scrub/common.h" #include "scrub/dabtree.h" #include "scrub/readdir.h" +#include "scrub/repair.h" /* Set us up to scrub directories. */ int xchk_setup_directory (struct xfs_scrub *sc) { +int error; + +if (xchk_could_repair (sc)) { + error =xrep_setup_directory (sc); +if (error) +return error; + } +return xchk_setup_inode_contents (sc,0 ); }
fs/xfs/scrub/dir_repair.c new file +964 -0
964 lines not on this page · raw message
fs/xfs/scrub/repair.h +16 -0
@@ -30,6 +30,16 @@ int xrep_init_btblock(struct xfs_scrub *sc, xfs_fsblock_t fsb, struct xfs_buf **bpp, xfs_btnum_t btnum, const struct xfs_buf_ops *ops); +static inline int +xrep_trans_commit ( +struct xfs_scrub *sc) +{ +int error =xfs_trans_commit (sc->tp); + + sc->tp =NULL ; +return error; +} +struct xbitmap ;int xrep_fix_freelist (struct xfs_scrub *sc,bool can_shrink);
@@ -57,6 +67,8 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,void xrep_force_quotacheck (struct xfs_scrub *sc,xfs_dqtype_t type);int xrep_ino_dqattach (struct xfs_scrub *sc); +int xrep_setup_directory (struct xfs_scrub *sc); +/* Metadata repairers */ int xrep_probe (struct xfs_scrub *sc);
@@ -64,6 +76,7 @@ int xrep_superblock(struct xfs_scrub *sc);int xrep_agf (struct xfs_scrub *sc);int xrep_agfl (struct xfs_scrub *sc);int xrep_agi (struct xfs_scrub *sc); +int xrep_directory (struct xfs_scrub *sc);#else
@@ -83,11 +96,14 @@ xrep_calc_ag_resblks( return 0; } +#define xrep_setup_directory (sc) (0) +#define xrep_probe xrep_notsupported#define xrep_superblock xrep_notsupported#define xrep_agf xrep_notsupported#define xrep_agfl xrep_notsupported#define xrep_agi xrep_notsupported +#define xrep_directory xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +1 -1
@@ -299,7 +299,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_directory, .scrub = xchk_directory, - .repair = xrep_notsupported, + .repair = xrep_directory, }, [XFS_SCRUB_TYPE_XATTR] = { /* extended attributes */ .type = ST_INODE,
fs/xfs/scrub/tempfile.c +42 -0
66 lines not on this page · raw message
fs/xfs/scrub/tempfile.h +2 -0
@@ -11,10 +11,12 @@ int xrep_tempfile_create(struct xfs_scrub *sc, uint16_t mode);void xrep_tempfile_rele (struct xfs_scrub *sc);bool xrep_tempfile_iolock_nowait (struct xfs_scrub *sc); +int xrep_tempfile_iolock_polled (struct xfs_scrub *sc);void xrep_tempfile_iounlock (struct xfs_scrub *sc);void xrep_tempfile_ilock (struct xfs_scrub *sc);bool xrep_tempfile_ilock_nowait (struct xfs_scrub *sc); +int xrep_tempfile_ilock_polled (struct xfs_scrub *sc);void xrep_tempfile_iunlock (struct xfs_scrub *sc);bool xrep_is_tempfile (const struct xfs_inode *ip);#else
fs/xfs/scrub/trace.c +1 -0
@@ -17,6 +17,7 @@#include "scrub/xfile.h" #include "scrub/xfarray.h" #include "scrub/iscan.h" +#include "xfs_da_format.h" /* Figure out which block the btree cursor was pointing to. */ static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +64 -0
70 lines not on this page · raw message
PATCH v11 2/3 xfs: add hooks to do directory updates
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
While we're scanning the filesystem, we still need to keep the tempdir up to date with whatever changes get made to the you know what.
--- fs/xfs/libxfs/xfs_dir2.c | 2 + fs/xfs/libxfs/xfs_dir2.h | 2 + fs/xfs/scrub/dir_repair.c | 90 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +1 -1
@@ -435,7 +435,7 @@ intxfs_dir_removename ( struct xfs_trans *tp, struct xfs_inode *dp, - struct xfs_name *name, + const struct xfs_name *name, xfs_ino_t ino, xfs_extlen_t total) /* bmap's total block count */ {
fs/xfs/libxfs/xfs_dir2.h +1 -1
@@ -46,7 +46,7 @@ extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp, const struct xfs_name *name, xfs_ino_t *inum, struct xfs_name *ci_name);extern int xfs_dir_removename (struct xfs_trans *tp,struct xfs_inode *dp, -struct xfs_name *name,xfs_ino_t ino, +const struct xfs_name *name,xfs_ino_t ino,xfs_extlen_t tot); extern int xfs_dir_replace(struct xfs_trans *tp, struct xfs_inode *dp, const struct xfs_name *name, xfs_ino_t inum,
fs/xfs/scrub/dir_repair.c +90 -0
122 lines not on this page · raw message
PATCH v11 3/3 xfs: compare generated and existing dirents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check our work to make sure we found all the dirents that the original directory had.
--- fs/xfs/scrub/dir_repair.c | 75 ++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/trace.h | 1 + 2 files changed, 74 insertions(+), 2 deletions(-)
fs/xfs/scrub/dir_repair.c +73 -2
@@ -746,7 +746,10 @@ xrep_dir_scan_dirtree(return 0 ; } -/* Dump a dirent from the temporary dir. */ +/* +* Dump a dirent from the temporary dir and check it against the dir we're +* rebuilding. We are not committing any of this. +*/ STATIC int xrep_dir_dump_tempdir( struct xfs_scrub *sc,
@@ -757,8 +760,10 @@ xrep_dir_dump_tempdir( void *priv) {struct xrep_dir *rd = priv; +xfs_ino_t child_ino;bool child_dirent = true; -int error =0 ; +bool compare_dirent = true; +int error;/* * The tempdir was created with a dotdot entry pointing to the root
@@ -781,11 +786,30 @@ xrep_dir_dump_tempdir( }if (xrep_dir_samename (name, &xfs_name_dot)) { child_dirent = false; + compare_dirent = false; ino = sc->ip->i_ino; }trace_xrep_dir_dumpname (sc->tempip, name, ino); +/* Check that the dir being repaired has the same entry. */ +if (compare_dirent) { + error =xchk_dir_lookup (sc, sc->ip, name, &child_ino); +if (error == -ENOENT ) { +trace_xrep_dir_checkname (sc->ip, name,NULLFSINO ); +ASSERT (error != -ENOENT ); +return -EFSCORRUPTED ; + } +if (error) +return error; + +if (ino != child_ino) { +trace_xrep_dir_checkname (sc->ip, name, child_ino); +ASSERT (ino == child_ino); +return -EFSCORRUPTED ; + } + } +/* * Set ourselves up to free every dirent in the tempdir because * directory inactivation won't do it for us. The rest of the online
@@ -801,6 +825,49 @@ xrep_dir_dump_tempdir(return error; } +/* +* Dump a dirent from the dir we're rebuilding and check it against the +* temporary dir. This assumes that the directory wasn't really corrupt to +* begin with. +*/ +STATIC int +xrep_dir_dump_baddir ( +struct xfs_scrub *sc, +struct xfs_inode *dp, +xfs_dir2_dataptr_t dapos, +const struct xfs_name *name, +xfs_ino_t ino, +void *priv) +{ +xfs_ino_t child_ino; +int error; + +/* Ignore the directory's dot and dotdot entries. */ +if (xrep_dir_samename (name, &xfs_name_dotdot) || +xrep_dir_samename (name, &xfs_name_dot)) +return 0 ; + +trace_xrep_dir_dumpname (sc->ip, name, ino); + +/* Check that the tempdir has the same entry. */ + error =xchk_dir_lookup (sc, sc->tempip, name, &child_ino); +if (error == -ENOENT ) { +trace_xrep_dir_checkname (sc->tempip, name,NULLFSINO ); +ASSERT (error != -ENOENT ); +return -EFSCORRUPTED ; + } +if (error) +return error; + +if (ino != child_ino) { +trace_xrep_dir_checkname (sc->tempip, name, child_ino); +ASSERT (ino == child_ino); +return -EFSCORRUPTED ; + } + +return 0 ; +} +/* * "Commit" the new directory structure to the file that we're repairing. *
@@ -886,6 +953,10 @@ xrep_dir_rebuild_tree(if (error)return error; + error =xchk_dir_walk (sc, sc->ip, xrep_dir_dump_baddir, rd); +if (error) +return error; + error =xchk_dir_walk (sc, sc->tempip, xrep_dir_dump_tempdir, rd);if (error)return error;
fs/xfs/scrub/trace.h +1 -0
@@ -1279,6 +1279,7 @@ DEFINE_XREP_DIRENT_CLASS(xrep_dir_createname);DEFINE_XREP_DIRENT_CLASS (xrep_dir_removename);DEFINE_XREP_DIRENT_CLASS (xrep_dir_replacename);DEFINE_XREP_DIRENT_CLASS (xrep_dir_dumpname); +DEFINE_XREP_DIRENT_CLASS (xrep_dir_checkname); DECLARE_EVENT_CLASS(xrep_dir_class, TP_PROTO(struct xfs_inode *dp, xfs_ino_t parent_ino),
v11 cover xfs: online checking of parent pointers
In reply to Darrick J. WongHi all,
Update the existing online parent pointer checker to confirm the directory entries that should also exist.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-check
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-parent-check
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_parent.c | 42 ++++ fs/xfs/libxfs/xfs_parent.h | 10 + fs/xfs/scrub/parent.c | 501 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 33 +++ 5 files changed, 587 insertions(+), 1 deletion(-)
PATCH v11 1/2 xfs: scrub parent pointers
From: Darrick J. Wong <[email protected]>
Actually check parent pointers now.
--- fs/xfs/scrub/parent.c | 280 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 280 insertions(+)
fs/xfs/scrub/parent.c +280 -0
@@ -14,9 +14,13 @@#include "xfs_icache.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/readdir.h" +#include "scrub/listxattr.h" +#include "scrub/trace.h" /* Set us up to scrub parents. */ int
@@ -231,6 +235,8 @@ xchk_parent_validate(return error; } +STATIC int xchk_parent_pptr(struct xfs_scrub *sc); +/* Scrub a parent pointer. */ int xchk_parent(
@@ -240,6 +246,9 @@ xchk_parent(xfs_ino_t parent_ino;int error; +if (xfs_has_parent (mp)) +return xchk_parent_pptr (sc); +/* * If we're a directory, check that the '..' link points up to * a directory that has one entry pointing to us.
@@ -282,3 +291,274 @@ xchk_parent(return xchk_parent_validate (sc, parent_ino); } + +/* +* Checking of Parent Pointers +* =========================== +* +* On filesystems with directory parent pointers, we check the referential +* integrity by visiting each parent pointer of a child file and checking that +* the directory referenced by the pointer actually has a dirent pointing back +* to the child file. +*/ + +struct xchk_pptrs { +struct xfs_scrub *sc; + +/* Scratch buffer for scanning pptr xattrs */ +struct xfs_parent_name_irec pptr; + +/* Parent of this directory. */ +xfs_ino_t parent_ino; +}; + +/* Look up the dotdot entry so that we can check it as we walk the pptrs. */ +STATIC int +xchk_parent_dotdot ( +struct xchk_pptrs *pp) +{ +struct xfs_scrub *sc = pp->sc; +int error; + +if (!S_ISDIR (VFS_I (sc->ip)->i_mode)) { + pp->parent_ino =NULLFSINO ; +return 0 ; + } + +/* Look up '..' */ + error =xchk_dir_lookup (sc, sc->ip, &xfs_name_dotdot, &pp->parent_ino); +if (!xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; +if (!xfs_verify_dir_ino (sc->mp, pp->parent_ino)) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +/* Is this the root dir? Then '..' must point to itself. */ +if (sc->ip == sc->mp->m_rootip && sc->ip->i_ino != pp->parent_ino) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); + +return 0 ; +} + +/* +* Try to lock a parent directory for checking dirents. Returns the inode +* flags for the locks we now hold, or zero if we failed. +*/ +STATIC unsigned int +xchk_parent_lock_dir ( +struct xfs_scrub *sc, +struct xfs_inode *dp) +{ +if (!xfs_ilock_nowait (dp,XFS_IOLOCK_SHARED )) +return 0 ; + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_SHARED )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +if (!xfs_need_iread_extents (&dp->i_df)) +return XFS_IOLOCK_SHARED |XFS_ILOCK_SHARED ; + +xfs_iunlock (dp,XFS_ILOCK_SHARED ); + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_EXCL )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +return XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; +} + +/* Check the forward link (dirent) associated with this parent pointer. */ +STATIC int +xchk_parent_dirent ( +struct xchk_pptrs *pp, +struct xfs_inode *dp) +{ +struct xfs_name xname = { + .name = pp->pptr.p_name, + .len = pp->pptr.p_namelen, + }; +struct xfs_scrub *sc = pp->sc; +xfs_ino_t child_ino; +int error; + +/* +* Use the name attached to this parent pointer to look up the +* directory entry in the alleged parent. +*/ + error =xchk_dir_lookup (sc, dp, &xname, &child_ino); +if (error == -ENOENT ) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* Does the inode number match? */ +if (child_ino != sc->ip->i_ino) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } + +/* +* If we're scanning a directory, we should only ever encounter a +* single parent pointer, and it should match the dotdot entry. We set +* the parent_ino from the dotdot entry before the scan, so compare it +* now. +*/ +if (!S_ISDIR (VFS_I (sc->ip)->i_mode)) +return 0 ; + +if (pp->parent_ino != dp->i_ino) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } + + pp->parent_ino =NULLFSINO ; +return 0 ; +} + +/* Try to grab a parent directory. */ +STATIC int +xchk_parent_iget ( +struct xchk_pptrs *pp, +struct xfs_inode **dpp) +{ +struct xfs_scrub *sc = pp->sc; +struct xfs_inode *ip; +int error; + +/* Validate inode number. */ + error =xfs_dir_ino_validate (sc->mp, pp->pptr.p_ino); +if (error) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + + error =xchk_iget (sc, pp->pptr.p_ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* The parent must be a directory. */ +if (!S_ISDIR (VFS_I (ip)->i_mode)) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + +/* Validate generation number. */ +if (VFS_I (ip)->i_generation != pp->pptr.p_gen) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + + *dpp = ip; +return 0 ; +out_rele: +xchk_irele (sc, ip); +return 0 ; +} + +/* +* Walk an xattr of a file. If this xattr is a parent pointer, follow it up +* to a parent directory and check that the parent has a dirent pointing back +* to us. +*/ +STATIC int +xchk_parent_scan_attr ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +void *priv) +{ +struct xchk_pptrs *pp = priv; +struct xfs_inode *dp =NULL ; +const struct xfs_parent_name_rec *rec = (const void *)name; +unsigned int lockmode; +int error; + +/* Ignore incomplete xattrs */ +if (attr_flags &XFS_ATTR_INCOMPLETE ) +return 0 ; + +/* Ignore anything that isn't a parent pointer. */ +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ; + +/* Does the ondisk parent pointer structure make sense? */ +if (!xfs_parent_namecheck (sc->mp, rec, namelen, attr_flags)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +if (!xfs_parent_valuecheck (sc->mp, value, valuelen)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +xfs_parent_irec_from_disk (&pp->pptr, rec, value, valuelen); + + error =xchk_parent_iget (pp, &dp); +if (error) +return error; +if (!dp) +return 0 ; + +/* Try to lock the inode. */ + lockmode =xchk_parent_lock_dir (sc, dp); +if (!lockmode) { +xchk_set_incomplete (sc); + error = -ECANCELED ; + goto out_rele; + } + + error =xchk_parent_dirent (pp, dp); +if (error) + goto out_unlock; + +out_unlock: +xfs_iunlock (dp, lockmode); +out_rele: +xchk_irele (sc, dp); +return error; +} + +/* Check parent pointers of a file. */ +STATIC int +xchk_parent_pptr ( +struct xfs_scrub *sc) +{ +struct xchk_pptrs *pp; +int error; + + pp =kvzalloc (sizeof (struct xchk_pptrs ),XCHK_GFP_FLAGS ); +if (!pp) +return -ENOMEM ; + pp->sc = sc; + + error =xchk_parent_dotdot (pp); +if (error) + goto out_pp; + + error =xchk_xattr_walk (sc, sc->ip, xchk_parent_scan_attr, pp); +if (error == -ECANCELED ) { + error =0 ; + goto out_pp; + } +if (error) + goto out_pp; + +out_pp: +kvfree (pp); +return error; +}
PATCH v11 2/2 xfs: deferred scrub of parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based dirent check fails, retain those parent pointers and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_parent.c | 42 ++++++++ fs/xfs/libxfs/xfs_parent.h | 10 ++ fs/xfs/scrub/parent.c | 229 +++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/trace.h | 33 ++++++ 5 files changed, 311 insertions(+), 5 deletions(-)
fs/xfs/Makefile +1 -1
@@ -169,6 +169,7 @@ xfs-y += $(addprefix scrub/, \ scrub.o \ symlink.o \ xfarray.o \ + xfblob.o \ xfile.o \ )
@@ -182,7 +183,6 @@ xfs-y += $(addprefix scrub/, \ dir_repair.o \ repair.o \ tempfile.o \ - xfblob.o \ ) endif endif
fs/xfs/libxfs/xfs_parent.c +42 -0
45 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +10 -0
@@ -101,4 +101,14 @@ void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec,void xfs_parent_irec_hashname (struct xfs_mount *mp,struct xfs_parent_name_irec *irec); +/* Scratchpad memory so that raw parent operations don't burn stack space. */ +struct xfs_parent_scratch { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +int xfs_parent_lookup (struct xfs_trans *tp,struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/parent.c +225 -4
273 lines not on this page · raw message
fs/xfs/scrub/trace.h +33 -0
@@ -954,6 +954,39 @@ TRACE_EVENT(xchk_nlinks_live_update, __get_str(name)) ); +DECLARE_EVENT_CLASS (xchk_pptr_class, + TP_PROTO(struct xfs_inode *ip, const unsigned char *name, + unsigned int namelen, xfs_ino_t parent_ino), +TP_ARGS (ip, name, namelen, parent_ino), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned int, namelen) + __dynamic_array(char, name, namelen) +__field (xfs_ino_t, parent_ino) + ), +TP_fast_assign ( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->namelen = namelen; + memcpy(__get_str(name), name, namelen); + __entry->parent_ino = parent_ino; + ), +TP_printk ("dev %d:%d ino 0x%llx name '%.*s' parent_ino 0x%llx" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->namelen, +__get_str (name), + __entry->parent_ino) +) +#define DEFINE_XCHK_PPTR_CLASS (name) \ +DEFINE_EVENT(xchk_pptr_class, name, \ + TP_PROTO(struct xfs_inode *ip, const unsigned char *name, \ + unsigned int namelen, xfs_ino_t parent_ino), \ + TP_ARGS(ip, name, namelen, parent_ino)) +DEFINE_XCHK_PPTR_CLASS (xchk_parent_defer); +DEFINE_XCHK_PPTR_CLASS (xchk_parent_slowpath); +/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
v11 cover xfs: online checking of parent pointers
In reply to Darrick J. WongHi all,
With this patchset, we implement online repairs for parent pointers. This is structured similarly to the directory repair code in that we scan the entire filesystem looking for dirents and use them to reconstruct the parent pointer information.
Note that the atomic swapext and block reaping code is NOT ported for this PoC, so we do not commit any repairs.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-repair
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-parent-repair
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_parent.c | 37 ++ fs/xfs/libxfs/xfs_parent.h | 8 fs/xfs/scrub/parent.c | 10 + fs/xfs/scrub/parent_repair.c | 739 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 4 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.c | 2 fs/xfs/scrub/trace.h | 74 ++++ 9 files changed, 876 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/scrub/parent_repair.c
PATCH v11 1/3 xfs: repair parent pointers by scanning directories
From: Darrick J. Wong <[email protected]>
Walk the filesystem to rebuild parent pointer information.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_parent.c | 18 + fs/xfs/libxfs/xfs_parent.h | 4 fs/xfs/scrub/parent.c | 10 + fs/xfs/scrub/parent_repair.c | 614 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 4 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.c | 2 fs/xfs/scrub/trace.h | 71 +++++ 9 files changed, 725 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/scrub/parent_repair.c
fs/xfs/Makefile +1 -0
@@ -181,6 +181,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ dir_repair.o \ + parent_repair.o \ repair.o \ tempfile.o \ )
fs/xfs/libxfs/xfs_parent.c +18 -0
@@ -357,3 +357,21 @@ xfs_parent_lookup(return xfs_attr_get_ilocked (&scr->args); } + +/* +* Attach the parent pointer (@pptr -> @name) to @ip immediately. Caller must +* not have a transaction or hold the ILOCK. The update will not use logged +* xattrs. This is for specialized repair functions only. The scratchpad need +* not be initialized. +*/ +int +xfs_parent_set ( +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scr) +{ +xfs_parent_irec_to_disk (&scr->rec, pptr); +xfs_parent_scratch_init (NULL , ip, pptr, scr); + +return xfs_attr_set (&scr->args); +}
fs/xfs/libxfs/xfs_parent.h +4 -0
@@ -111,4 +111,8 @@ int xfs_parent_lookup(struct xfs_trans *tp, struct xfs_inode *ip,const struct xfs_parent_name_irec *pptr, struct xfs_parent_scratch *scratch); +int xfs_parent_set (struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/parent.c +10 -0
@@ -10,6 +10,7 @@#include "xfs_trans_resv.h" #include "xfs_mount.h" #include "xfs_log_format.h" +#include "xfs_trans.h" #include "xfs_inode.h" #include "xfs_icache.h" #include "xfs_dir2.h"
@@ -24,12 +25,21 @@#include "scrub/xfarray.h" #include "scrub/xfblob.h" #include "scrub/trace.h" +#include "scrub/repair.h" /* Set us up to scrub parents. */ int xchk_setup_parent (struct xfs_scrub *sc) { +int error; + +if (xchk_could_repair (sc)) { + error =xrep_setup_parent (sc); +if (error) +return error; + } +return xchk_setup_inode_contents (sc,0 ); }
fs/xfs/scrub/parent_repair.c new file +614 -0
614 lines not on this page · raw message
fs/xfs/scrub/repair.h +4 -0
@@ -68,6 +68,7 @@ void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);int xrep_ino_dqattach (struct xfs_scrub *sc);int xrep_setup_directory (struct xfs_scrub *sc); +int xrep_setup_parent (struct xfs_scrub *sc);/* Metadata repairers */
@@ -77,6 +78,7 @@ int xrep_agf(struct xfs_scrub *sc);int xrep_agfl (struct xfs_scrub *sc);int xrep_agi (struct xfs_scrub *sc);int xrep_directory (struct xfs_scrub *sc); +int xrep_parent (struct xfs_scrub *sc);#else
@@ -97,6 +99,7 @@ xrep_calc_ag_resblks( }#define xrep_setup_directory (sc) (0) +#define xrep_setup_parent (sc) (0)#define xrep_probe xrep_notsupported#define xrep_superblock xrep_notsupported
@@ -104,6 +107,7 @@ xrep_calc_ag_resblks(#define xrep_agfl xrep_notsupported#define xrep_agi xrep_notsupported#define xrep_directory xrep_notsupported +#define xrep_parent xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +1 -1
@@ -317,7 +317,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_parent, .scrub = xchk_parent, - .repair = xrep_notsupported, + .repair = xrep_parent, }, [XFS_SCRUB_TYPE_RTBITMAP] = { /* realtime bitmap */ .type = ST_FS,
fs/xfs/scrub/trace.c +2 -0
@@ -18,6 +18,8 @@#include "scrub/xfarray.h" #include "scrub/iscan.h" #include "xfs_da_format.h" +#include "xfs_da_btree.h" +#include "xfs_parent.h" /* Figure out which block the btree cursor was pointing to. */ static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +71 -0
83 lines not on this page · raw message
PATCH v11 2/3 xfs: repair parent pointers with live scan hooks
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Use the nlink hooks to keep our tempfile's parent pointers up to date.
--- fs/xfs/libxfs/xfs_parent.c | 19 ++++++++ fs/xfs/libxfs/xfs_parent.h | 4 ++ fs/xfs/scrub/parent_repair.c | 100 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 2 + 4 files changed, 125 insertions(+)
fs/xfs/libxfs/xfs_parent.c +19 -0
@@ -375,3 +375,22 @@ xfs_parent_set(return xfs_attr_set (&scr->args); } + +/* +* Remove the parent pointer (@rec -> @name) from @ip immediately. Caller must +* not have a transaction or hold the ILOCK. The update will not use logged +* xattrs. This is for specialized repair functions only. The scratchpad need +* not be initialized. +*/ +int +xfs_parent_unset ( +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scr) +{ +xfs_parent_irec_to_disk (&scr->rec, pptr); +xfs_parent_scratch_init (NULL , ip, pptr, scr); + scr->args.op_flags |=XFS_DA_OP_REMOVE ; + +return xfs_attr_set (&scr->args); +}
fs/xfs/libxfs/xfs_parent.h +4 -0
@@ -115,4 +115,8 @@ int xfs_parent_set(struct xfs_inode *ip,const struct xfs_parent_name_irec *pptr, struct xfs_parent_scratch *scratch); +int xfs_parent_unset (struct xfs_inode *ip, +const struct xfs_parent_name_irec *rec, +struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/parent_repair.c +100 -0
144 lines not on this page · raw message
fs/xfs/scrub/trace.h +2 -0
@@ -1372,6 +1372,7 @@ DEFINE_EVENT(xrep_pptr_class, name, \ TP_PROTO(struct xfs_inode *ip, const struct xfs_parent_name_irec *pptr), \ TP_ARGS(ip, pptr)) DEFINE_XREP_PPTR_CLASS(xrep_pptr_createname); +DEFINE_XREP_PPTR_CLASS (xrep_pptr_removename);DEFINE_XREP_PPTR_CLASS (xrep_pptr_dumpname); DECLARE_EVENT_CLASS(xrep_pptr_scan_class,
@@ -1408,6 +1409,7 @@ DEFINE_EVENT(xrep_pptr_scan_class, name, \ const struct xfs_name *name), \ TP_ARGS(ip, dp, name)) DEFINE_XREP_PPTR_SCAN_CLASS(xrep_pptr_add_pointer); +DEFINE_XREP_PPTR_SCAN_CLASS (xrep_pptr_remove_pointer);#endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
PATCH v11 3/3 xfs: compare generated and existing parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check our work to make sure we found all the parent pointers that the original file had.
--- fs/xfs/scrub/parent_repair.c | 29 +++++++++++++++++++++++++++-- fs/xfs/scrub/trace.h | 1 + 2 files changed, 28 insertions(+), 2 deletions(-)
fs/xfs/scrub/parent_repair.c +27 -2
@@ -486,7 +486,10 @@ xrep_pptr_scan_dirtree(return 0 ; } -/* Dump a parent pointer from the temporary file. */ +/* +* Dump a parent pointer from the temporary file and check it against the file +* we're rebuilding. We are not committing any of this. +*/ STATIC int xrep_pptr_dump_tempptr( struct xfs_scrub *sc,
@@ -500,6 +503,8 @@ xrep_pptr_dump_tempptr( {struct xrep_pptrs *rp = priv;const struct xfs_parent_name_rec *rec = (const void *)name; +struct xfs_inode *other_ip; +int error;if (!(attr_flags &XFS_ATTR_PARENT ))return 0 ;
@@ -508,10 +513,26 @@ xrep_pptr_dump_tempptr( !xfs_parent_valuecheck(sc->mp, value, valuelen)) return -EFSCORRUPTED; +if (ip == sc->ip) + other_ip = sc->tempip; +else if (ip == sc->tempip) + other_ip = sc->ip; +else +return -EFSCORRUPTED ; +xfs_parent_irec_from_disk (&rp->pptr, rec, value, valuelen);trace_xrep_pptr_dumpname (sc->tempip, &rp->pptr); -return 0 ; + + error =xfs_parent_lookup (sc->tp, other_ip, &rp->pptr, + &rp->pptr_scratch); +if (error == -ENOATTR ) { +trace_xrep_pptr_checkname (other_ip, &rp->pptr); +ASSERT (error != -ENOATTR ); +return -EFSCORRUPTED ; + } + +return error; }/*
@@ -590,6 +611,10 @@ xrep_pptr_rebuild_tree(if (error)return error; + error =xchk_xattr_walk (sc, sc->ip, xrep_pptr_dump_tempptr, rp); +if (error) +return error; +return xchk_xattr_walk (sc, sc->tempip, xrep_pptr_dump_tempptr, rp); }
fs/xfs/scrub/trace.h +1 -0
@@ -1374,6 +1374,7 @@ DEFINE_EVENT(xrep_pptr_class, name, \DEFINE_XREP_PPTR_CLASS (xrep_pptr_createname);DEFINE_XREP_PPTR_CLASS (xrep_pptr_removename);DEFINE_XREP_PPTR_CLASS (xrep_pptr_dumpname); +DEFINE_XREP_PPTR_CLASS (xrep_pptr_checkname); DECLARE_EVENT_CLASS(xrep_pptr_scan_class, TP_PROTO(struct xfs_inode *ip, const struct xfs_inode *dp,
v11 cover xfs: online checking of directories
In reply to Darrick J. WongHi all,
Update the existing online directory checker to confirm the parent pointers that should also exist.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-check
--- fs/xfs/scrub/dir.c | 338 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 2 2 files changed, 339 insertions(+), 1 deletion(-)
PATCH v11 1/2 xfs: check dirents have parent pointers
From: Darrick J. Wong <[email protected]>
If the fs has parent pointers, we need to check that each child dirent points to a file that has a parent pointer pointing back at us.
--- fs/xfs/scrub/dir.c | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+), 1 deletion(-)
fs/xfs/scrub/dir.c +116 -1
@@ -15,6 +15,8 @@#include "xfs_icache.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/dabtree.h"
@@ -39,6 +41,20 @@ xchk_setup_directory(/* Directories */ +struct xchk_dir { +struct xfs_scrub *sc; + +/* Scratch buffer for scanning pptr xattrs */ +struct xfs_parent_name_irec pptr; + +/* xattr key and da args for parent pointer validation. */ +struct xfs_parent_scratch pptr_scratch; + +/* Name buffer for dirent revalidation. */ +uint8_t namebuf[MAXNAMELEN ]; + +}; +/* Scrub a directory entry. */ /* Check that an inode's mode matches a given XFS_DIR3_FT_* type. */
@@ -61,6 +77,88 @@ xchk_dir_check_ftype(xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); } +/* +* Try to lock a child file for checking parent pointers. Returns the inode +* flags for the locks we now hold, or zero if we failed. +*/ +STATIC unsigned int +xchk_dir_lock_child ( +struct xfs_scrub *sc, +struct xfs_inode *ip) +{ +if (!xfs_ilock_nowait (ip,XFS_IOLOCK_SHARED )) +return 0 ; + +if (!xfs_ilock_nowait (ip,XFS_ILOCK_SHARED )) { +xfs_iunlock (ip,XFS_IOLOCK_SHARED ); +return 0 ; + } + +if (!xfs_inode_has_attr_fork (ip) || !xfs_need_iread_extents (&ip->i_af)) +return XFS_IOLOCK_SHARED |XFS_ILOCK_SHARED ; + +xfs_iunlock (ip,XFS_ILOCK_SHARED ); + +if (!xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) { +xfs_iunlock (ip,XFS_IOLOCK_SHARED ); +return 0 ; + } + +return XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; +} + +/* Check the backwards link (parent pointer) associated with this dirent. */ +STATIC int +xchk_dir_parent_pointer ( +struct xchk_dir *sd, +const struct xfs_name *name, +struct xfs_inode *ip) +{ +struct xfs_scrub *sc = sd->sc; +int error; + + sd->pptr.p_ino = sc->ip->i_ino; + sd->pptr.p_gen =VFS_I (sc->ip)->i_generation; + sd->pptr.p_namelen = name->len; +memcpy (sd->pptr.p_name, name->name, name->len); +xfs_parent_irec_hashname (sc->mp, &sd->pptr); + + error =xfs_parent_lookup (sc->tp, ip, &sd->pptr, &sd->pptr_scratch); +if (error == -ENOATTR ) +xchk_fblock_xref_set_corrupt (sc,XFS_DATA_FORK ,0 ); + +return 0 ; +} + +/* Look for a parent pointer matching this dirent, if the child isn't busy. */ +STATIC int +xchk_dir_check_pptr_fast ( +struct xchk_dir *sd, +xfs_dir2_dataptr_t dapos, +const struct xfs_name *name, +struct xfs_inode *ip) +{ +struct xfs_scrub *sc = sd->sc; +unsigned int lockmode; +int error; + +/* dot and dotdot entries do not have parent pointers */ +if (!strncmp ("." , name->name, name->len) || + !strncmp (".." , name->name, name->len)) +return 0 ; + +/* Try to lock the inode. */ + lockmode =xchk_dir_lock_child (sc, ip); +if (!lockmode) { +xchk_set_incomplete (sc); +return -ECANCELED ; + } + + error =xchk_dir_parent_pointer (sd, name, ip); +xfs_iunlock (ip, lockmode); +return error; +} +/* * Scrub a single directory entry. *
@@ -78,6 +176,7 @@ xchk_dir_actor( {struct xfs_mount *mp = dp->i_mount;struct xfs_inode *ip; +struct xchk_dir *sd = priv;xfs_ino_t lookup_ino;xfs_dablk_t offset;int error =0 ;
@@ -144,6 +243,14 @@ xchk_dir_actor( goto out;xchk_dir_check_ftype (sc, offset, ip, name->type); + +if (xfs_has_parent (mp)) { + error =xchk_dir_check_pptr_fast (sd, dapos, name, ip); +if (error) + goto out_rele; + } + +out_rele:xchk_irele (sc, ip); out:if (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT )
@@ -759,6 +866,7 @@ intxchk_directory ( struct xfs_scrub *sc) { +struct xchk_dir *sd;int error;if (!S_ISDIR (VFS_I (sc->ip)->i_mode))
@@ -786,9 +894,16 @@ xchk_directory(if (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT )return 0 ; + sd =kvzalloc (sizeof (struct xchk_dir ),XCHK_GFP_FLAGS ); +if (!sd) +return -ENOMEM ; + sd->sc = sc; +/* Look up every name in this directory by hash. */ - error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor,NULL ); + error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor, sd);if (error == -ECANCELED ) error =0 ; + +kvfree (sd);return error; }
PATCH v11 2/2 xfs: deferred scrub of dirents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based parent pointer check fails, retain those dirents and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/scrub/dir.c | 225 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 2 2 files changed, 225 insertions(+), 2 deletions(-)
fs/xfs/scrub/dir.c +223 -2
@@ -22,6 +22,10 @@#include "scrub/dabtree.h" #include "scrub/readdir.h" #include "scrub/repair.h" +#include "scrub/trace.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" +#include "scrub/xfblob.h" /* Set us up to scrub directories. */ int
@@ -41,6 +45,18 @@ xchk_setup_directory(/* Directories */ +/* Deferred directory entry that we saved for later. */ +struct xchk_dirent { +/* Cookie for retrieval of the dirent name. */ +xfblob_cookie name_cookie; + +/* Child inode number. */ +xfs_ino_t ino; + +/* Length of the pptr name. */ +uint8_t namelen; +}; +struct xchk_dir {struct xfs_scrub *sc;
@@ -50,6 +66,15 @@ struct xchk_dir {/* xattr key and da args for parent pointer validation. */ struct xfs_parent_scratch pptr_scratch; +/* Fixed-size array of xchk_dirent structures. */ +struct xfarray *dir_entries; + +/* Blobs containing dirent names. */ +struct xfblob *dir_names; + +/* If we've cycled the ILOCK, we must revalidate deferred dirents. */ +bool need_revalidate; +/* Name buffer for dirent revalidation. */ uint8_t namebuf[MAXNAMELEN ];
@@ -150,8 +175,24 @@ xchk_dir_check_pptr_fast(/* Try to lock the inode. */ lockmode =xchk_dir_lock_child (sc, ip);if (!lockmode) { -xchk_set_incomplete (sc); -return -ECANCELED ; +struct xchk_dirent save_de = { + .namelen = name->len, + .ino = ip->i_ino, + }; + +/* Couldn't lock the inode, so save the dirent for later. */ +trace_xchk_dir_defer (sc->ip, name->name, name->len, ip->i_ino); + + error =xfblob_store (sd->dir_names, &save_de.name_cookie, + name->name, name->len); +if (xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; + + error =xfarray_append (sd->dir_entries, &save_de); +if (xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; + +return 0 ; } error =xchk_dir_parent_pointer (sd, name, ip);
@@ -861,6 +902,156 @@ xchk_directory_blocks(return error; } +/* +* Revalidate a dirent that we collected in the past but couldn't check because +* of lock contention. Returns 0 if the dirent is still valid, -ENOENT if it +* has gone away on us, or a negative errno. +*/ +STATIC int +xchk_dir_revalidate_dirent ( +struct xchk_dir *sd, +const struct xfs_name *xname, +xfs_ino_t ino) +{ +struct xfs_scrub *sc = sd->sc; +xfs_ino_t child_ino; +int error; + +/* +* Look up the directory entry. If we get -ENOENT, the directory entry +* went away and there's nothing to revalidate. Return any other +* error. +*/ + error =xchk_dir_lookup (sc, sc->ip, xname, &child_ino); +if (error) +return error; + +/* The inode number changed, nothing to revalidate. */ +if (ino != child_ino) +return -ENOENT ; + +return 0 ; +} + +/* +* Check a directory entry's parent pointers the slow way, which means we cycle +* locks a bunch and put up with revalidation until we get it done. +*/ +STATIC int +xchk_dir_slow_dirent ( +struct xchk_dir *sd, +struct xchk_dirent *dirent) +{ +struct xfs_name xname = { + .name = sd->namebuf, + .len = dirent->namelen, + }; +struct xfs_scrub *sc = sd->sc; +struct xfs_inode *ip; +unsigned int lockmode; +int error; + +/* Check that the deferred dirent still exists. */ +if (sd->need_revalidate) { + error =xchk_dir_revalidate_dirent (sd, &xname, dirent->ino); +if (error == -ENOENT ) +return 0 ; +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + } + + error =xchk_iget (sc, dirent->ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* +* If we can grab both IOLOCK and ILOCK of the alleged child, we can +* proceed with the validation. +*/ + lockmode =xchk_dir_lock_child (sc, ip); +if (lockmode) + goto check_pptr; + +/* +* We couldn't lock the child file. Drop all the locks and try to +* get them again, one at a time. +*/ +xchk_iunlock (sc, sc->ilock_flags); + sd->need_revalidate = true; + +trace_xchk_dir_slowpath (sc->ip, xname.name, xname.len, ip->i_ino); + +while (true) { +xchk_ilock (sc,XFS_IOLOCK_EXCL ); +if (xfs_ilock_nowait (ip,XFS_IOLOCK_SHARED )) { +xchk_ilock (sc,XFS_ILOCK_EXCL ); +if (xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) { +break ; + } +xchk_iunlock (sc,XFS_ILOCK_EXCL ); + } +xchk_iunlock (sc,XFS_IOLOCK_EXCL ); + +if (xchk_should_terminate (sc, &error)) + goto out_rele; + +delay (1 ); + } + lockmode =XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; + +/* Revalidate, since we just cycled the locks. */ + error =xchk_dir_revalidate_dirent (sd, &xname, dirent->ino); +if (error == -ENOENT ) + goto out_unlock; +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , &error)) + goto out_unlock; + +check_pptr: + error =xchk_dir_parent_pointer (sd, &xname, ip); +out_unlock: +xfs_iunlock (ip, lockmode); +out_rele: +xchk_irele (sc, ip); +return error; +} + +/* Check all the dirents that we deferred the first time around. */ +STATIC int +xchk_dir_finish_slow_dirents ( +struct xchk_dir *sd) +{ +xfarray_idx_t array_cur; +int error; + + foreach_xfarray_idx(sd->dir_entries, array_cur) { +struct xchk_dirent dirent; + +if (sd->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) +return 0 ; + + error =xfarray_load (sd->dir_entries, array_cur, &dirent); +if (error) +return error; + + error =xfblob_load (sd->dir_names, dirent.name_cookie, + sd->namebuf, dirent.namelen); +if (error) +return error; + sd->namebuf[MAXNAMELEN -1 ] =0 ; + + error =xchk_dir_slow_dirent (sd, &dirent); +if (error) +return error; + } + +return 0 ; +} +/* Scrub a whole directory. */ int xchk_directory(
@@ -899,11 +1090,41 @@ xchk_directory(return -ENOMEM ; sd->sc = sc; +if (xfs_has_parent (sc->mp)) { +/* +* Set up some staging memory for dirents that we can't check +* due to locking contention. +*/ + error =xfarray_create (sc->mp,"directory entries" ,0 , +sizeof (struct xchk_dirent ), &sd->dir_entries); +if (error) + goto out_sd; + + error =xfblob_create (sc->mp,"dirent names" , &sd->dir_names); +if (error) + goto out_entries; + } +/* Look up every name in this directory by hash. */ error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor, sd);if (error == -ECANCELED ) error =0 ; +if (error) + goto out_names; +if (xfs_has_parent (sc->mp)) { + error =xchk_dir_finish_slow_dirents (sd); +if (error) + goto out_names; + } + +out_names: +if (sd->dir_names) +xfblob_destroy (sd->dir_names); +out_entries: +if (sd->dir_entries) +xfarray_destroy (sd->dir_entries); +out_sd:kvfree (sd);return error; }
fs/xfs/scrub/trace.h +2 -0
@@ -987,6 +987,8 @@ DEFINE_EVENT(xchk_pptr_class, name, \ TP_ARGS(ip, name, namelen, parent_ino))DEFINE_XCHK_PPTR_CLASS (xchk_parent_defer);DEFINE_XCHK_PPTR_CLASS (xchk_parent_slowpath); +DEFINE_XCHK_PPTR_CLASS (xchk_dir_defer); +DEFINE_XCHK_PPTR_CLASS (xchk_dir_slowpath);/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
v11 cover xfs: name-value xattr lookups
In reply to Darrick J. WongHi all,
Directory parent pointers are stored as namespaced extended attributes of a file. Because parent pointers can consume up to 267 bytes of space and xattr names are 255 bytes at most, we cannot use the usual attr name lookup functions to find a parent pointer. This is solvable by introducing a new lookup mode that checks both the name and the value of the xattr.
Therefore, introduce this new lookup mode. Because all parent pointer updates are logged, we must extend the xattr logging code to capture the VLOOKUP variants, and restore them when recovering logged operations. These new log formats are protected by the sb_incompat PARENT flag, so they do not need a separate log_incompat feature flag.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-attr-nvlookups
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-attr-nvlookups
--- db/attrset.c | 4 + libxfs/xfs_attr.c | 57 ++++++++++++++----- libxfs/xfs_attr.h | 9 ++- libxfs/xfs_attr_leaf.c | 45 +++++++++++++-- libxfs/xfs_da_btree.h | 10 +++ libxfs/xfs_log_format.h | 30 +++++++++- logprint/log_redo.c | 138 ++++++++++++++++++++++++++++++++++++++--------- logprint/logprint.h | 6 +- 8 files changed, 239 insertions(+), 60 deletions(-)
PATCH v11 1/10 xfs: make xfs_attr_set require XFS_DA_OP_REMOVE
From: Darrick J. Wong <[email protected]>
Quite a few patches from now, we're going to change the parent pointer xattr format to encode as much of the dirent name in the xattr name as fits, and spill the rest of it to the xattr value. To make this work correctly, we'll be adding the ability to look up xattrs based on name /and/ value.
Internally, the xattr data structure supports attributes with a zero length value, which is how we're going to store parent pointers for short dirent names. The parent pointer repair code uses xfs_attr_set to add missing and remove dangling parent pointers, so that interface must be capable of setting an xattr with args->value == NULL.
The userspace API doesn't support this, so xfs_attr_set currently treats a NULL args->value as a request to remove an attr. However, that's a quirk of the existing callers and the interface. Make the callers of xfs_attr_set to declare explicitly that they want to remove an xattr.
--- db/attrset.c | 4 +++- libxfs/xfs_attr.c | 9 +++++---- 2 files changed, 8 insertions(+), 5 deletions(-)
db/attrset.c +3 -1
@@ -184,7 +184,9 @@ attr_remove_f(int argc, char **argv) { - struct xfs_da_args args = { }; + struct xfs_da_args args = { + .op_flags =XFS_DA_OP_REMOVE , + };int c;if (cur_typ ==NULL ) {
libxfs/xfs_attr.c +5 -4
@@ -965,6 +965,7 @@ xfs_attr_set(struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres;bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool is_remove = args->op_flags &XFS_DA_OP_REMOVE ;int error, local;int rmt_blks =0 ;unsigned int total;
@@ -989,7 +990,7 @@ xfs_attr_set( args->op_flags =XFS_DA_OP_OKNOENT | (args->op_flags &XFS_DA_OP_LOGGED ); -if (args->value) { +if (!is_remove) {XFS_STATS_INC (mp, xs_attr_set); args->total =xfs_attr_calc_size (args, &local);
@@ -1023,7 +1024,7 @@ xfs_attr_set(if (error)return error; -if (args->value ||xfs_inode_hasattr (dp)) { +if (!is_remove ||xfs_inode_hasattr (dp)) { error =xfs_iext_count_may_overflow (dp,XFS_ATTR_FORK ,XFS_IEXT_ATTR_MANIP_CNT (rmt_blks));if (error == -EFBIG )
@@ -1037,7 +1038,7 @@ xfs_attr_set(switch (error) {case -EEXIST :/* if no value, we are performing a remove operation */ -if (!args->value) { +if (is_remove) { error =xfs_attr_defer_remove (args);break ; }
@@ -1049,7 +1050,7 @@ xfs_attr_set(break ;case -ENOATTR :/* Can't remove what isn't there. */ -if (!args->value) +if (is_remove) goto out_trans_cancel;/* Pure replace fails if no existing attr to replace. */
PATCH v11 2/10 xfs: allow xattr matching on name and value for local/sf attrs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a new NVLOOKUP flag to signal that the caller wants to look up an extended attribute by name and value. This only works with shortform and local attributes. Only parent pointers need this functionality and parent pointers cannot be remote xattrs, so this limitation is ok for now.
--- libxfs/xfs_attr_leaf.c | 45 +++++++++++++++++++++++++++++++++++++-------- libxfs/xfs_da_btree.h | 4 +++- 2 files changed, 40 insertions(+), 9 deletions(-)
libxfs/xfs_attr_leaf.c +37 -8
@@ -470,10 +470,12 @@ xfs_attr3_leaf_read(*/ static bool xfs_attr_match ( -struct xfs_da_args *args, -uint8_t namelen, -unsigned char *name, -int flags) +const struct xfs_da_args *args, +uint8_t namelen, +const unsigned char *name, +unsigned int valuelen, +const void *value, +int flags) {if (args->namelen != namelen)
@@ -481,6 +483,23 @@ xfs_attr_match(if (memcmp (args->name, name, namelen) !=0 )return false; +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) { +if (args->valuelen != valuelen) +return false; +if (args->valuelen && !value) { +/* not implemented for remote values */ +ASSERT (0 ); +return false; + } +if (valuelen && !args->value) { +/* caller gave us valuelen > 0 but no value?? */ +ASSERT (0 ); +return false; + } +if (valuelen >0 &&memcmp (args->value, value, valuelen) !=0 ) +return false; + } +/* Recovery ignores the INCOMPLETE flag. */ if ((args->op_flags &XFS_DA_OP_RECOVERY ) && args->attr_filter == (flags &XFS_ATTR_NSP_ONDISK_MASK ))
@@ -499,6 +518,10 @@ xfs_attr_copy_value( unsigned char *value, int valuelen) { +/* vlookups already supplied the attr value; don't copy anything */ +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) +return 0 ; +/* * No copy if all we have to do is get the length */
@@ -723,6 +746,7 @@ xfs_attr_sf_findname( base += size, i++) { size =xfs_attr_sf_entsize (sfe);if (!xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))continue ;break ;
@@ -891,6 +915,7 @@ xfs_attr_shortform_lookup(xfs_da_args_t *args)for (i =0 ; i < sf->hdr.count; sfe =xfs_attr_sf_nextentry (sfe), i++) {if (xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))return -EEXIST ; }
@@ -918,6 +943,7 @@ xfs_attr_shortform_getvalue(for (i =0 ; i < sf->hdr.count; sfe =xfs_attr_sf_nextentry (sfe), i++) {if (xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))return xfs_attr_copy_value (args, &sfe->nameval[args->namelen], sfe->valuelen);
@@ -970,7 +996,7 @@ xfs_attr_shortform_to_leaf( nargs.total = args->total; nargs.whichfork =XFS_ATTR_FORK ; nargs.trans = args->trans; - nargs.op_flags =XFS_DA_OP_OKNOENT ; + nargs.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_NVLOOKUP ; sfe = &sf->list[0 ];for (i =0 ; i < sf->hdr.count; i++) {
@@ -1180,7 +1206,7 @@ xfs_attr3_leaf_to_shortform( nargs.total = args->total; nargs.whichfork =XFS_ATTR_FORK ; nargs.trans = args->trans; - nargs.op_flags =XFS_DA_OP_OKNOENT ; + nargs.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_NVLOOKUP ;for (i =0 ; i < ichdr.count; entry++, i++) {if (entry->flags &XFS_ATTR_INCOMPLETE )
@@ -2479,14 +2505,17 @@ xfs_attr3_leaf_lookup_int(if (entry->flags &XFS_ATTR_LOCAL ) { name_loc =xfs_attr3_leaf_name_local (leaf, probe);if (!xfs_attr_match (args, name_loc->namelen, - name_loc->nameval, entry->flags)) + name_loc->nameval, +be16_to_cpu (name_loc->valuelen), + &name_loc->nameval[name_loc->namelen], + entry->flags))continue ; args->index = probe;return -EEXIST ; }else { name_rmt =xfs_attr3_leaf_name_remote (leaf, probe);if (!xfs_attr_match (args, name_rmt->namelen, - name_rmt->name, entry->flags)) + name_rmt->name,0 ,NULL , entry->flags))continue ; args->index = probe; args->rmtvaluelen =be32_to_cpu (name_rmt->valuelen);
libxfs/xfs_da_btree.h +3 -1
@@ -93,6 +93,7 @@ typedef struct xfs_da_args {#define XFS_DA_OP_REMOVE (1u << 6)/* this is a remove operation */ #define XFS_DA_OP_RECOVERY (1u << 7)/* Log recovery operation */ #define XFS_DA_OP_LOGGED (1u << 8)/* Use intent items to track op */ +#define XFS_DA_OP_NVLOOKUP (1u << 9)/* Match local attr on name+value */ #define XFS_DA_OP_FLAGS \ { XFS_DA_OP_JUSTCHECK, "JUSTCHECK" }, \
@@ -103,7 +104,8 @@ typedef struct xfs_da_args { { XFS_DA_OP_NOTIME, "NOTIME" }, \ { XFS_DA_OP_REMOVE, "REMOVE" }, \ { XFS_DA_OP_RECOVERY, "RECOVERY" }, \ - { XFS_DA_OP_LOGGED, "LOGGED" } + { XFS_DA_OP_LOGGED, "LOGGED" }, \ + { XFS_DA_OP_NVLOOKUP, "NVLOOKUP" }/* * Storage for holding state during Btree searches and split/join ops.
PATCH v11 3/10 xfs: preserve NVLOOKUP in xfs_attr_set
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Preserve the attr-value lookup flag when calling xfs_attr_set. Normal xattr users will never use this, but parent pointer fsck will.
--- libxfs/xfs_attr.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
libxfs/xfs_attr.c +4 -4
@@ -984,11 +984,11 @@ xfs_attr_set(/* * We have no control over the attribute names that userspace passes us * to remove, so we have to allow the name lookup prior to attribute -* removal to fail as well. Preserve the logged flag, since we need -* to pass that through to the logging code. +* removal to fail as well. Preserve the logged and vlookup flags, +* since we need to pass them through to the lower levels. */ - args->op_flags =XFS_DA_OP_OKNOENT | - (args->op_flags &XFS_DA_OP_LOGGED ); + args->op_flags &= (XFS_DA_OP_LOGGED |XFS_DA_OP_NVLOOKUP ); + args->op_flags |=XFS_DA_OP_OKNOENT ;if (!is_remove) {XFS_STATS_INC (mp, xs_attr_set);
PATCH v11 4/10 xfs: restructure xfs_attr_complete_op a bit
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Reduce the indentation in this function by flattening the nested if statements. We're going to add more code later to this function later, hence the early cleanup. No functional changes.
--- libxfs/xfs_attr.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
libxfs/xfs_attr.c +5 -5
@@ -419,11 +419,11 @@ xfs_attr_complete_op(bool do_replace = args->op_flags &XFS_DA_OP_REPLACE ; args->op_flags &= ~XFS_DA_OP_REPLACE ; -if (do_replace) { - args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; -return replace_state; - } -return XFS_DAS_DONE ; +if (!do_replace) +return XFS_DAS_DONE ; + + args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; +return replace_state; } static int
PATCH v11 5/10 xfs: use helpers to extract xattr op from opflags
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create helper functions to extract the xattr op from the ondisk xattri log item and the incore attr intent item. These will get more use in the patches that follow.
--- libxfs/xfs_attr.h | 5 +++++ 1 file changed, 5 insertions(+)
libxfs/xfs_attr.h +5 -0
@@ -529,6 +529,11 @@ struct xfs_attr_intent {struct xfs_bmbt_irec xattri_map; }; +static inline unsigned int +xfs_attr_intent_op (const struct xfs_attr_intent *attr) +{ +return attr->xattri_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +}/*======================================================================== * Function prototypes for the kernel.
PATCH v11 6/10 xfs: log NVLOOKUP xattr removal operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If high level code wants to do a deferred xattr remove operation with the NVLOOKUP flag set, we need to push this through the log.
--- libxfs/xfs_attr.c | 6 +++++- libxfs/xfs_log_format.h | 1 + 2 files changed, 6 insertions(+), 1 deletion(-)
libxfs/xfs_attr.c +5 -1
@@ -940,9 +940,13 @@ xfs_attr_defer_remove( {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_REMOVE ;int error; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_REMOVE , &new); +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVREMOVE ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
libxfs/xfs_log_format.h +1 -0
@@ -957,6 +957,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_SET 1/* Set the attribute */ #define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ +#define XFS_ATTRI_OP_FLAGS_NVREMOVE 4/* Remove attr w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
PATCH v11 7/10 xfs: log NVLOOKUP xattr setting operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If high level code wants to do a deferred xattr set operation with the NVLOOKUP flag set, we need to push this through the log.
--- libxfs/xfs_attr.c | 6 +++++- libxfs/xfs_log_format.h | 1 + 2 files changed, 6 insertions(+), 1 deletion(-)
libxfs/xfs_attr.c +5 -1
@@ -901,9 +901,13 @@ xfs_attr_defer_add( struct xfs_da_args *args) {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_SET ;int error =0 ; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_SET , &new); +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVSET ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
libxfs/xfs_log_format.h +1 -0
@@ -958,6 +958,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ #define XFS_ATTRI_OP_FLAGS_NVREMOVE 4/* Remove attr w/ vlookup */ +#define XFS_ATTRI_OP_FLAGS_NVSET 5/* Set attr with w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
PATCH v11 8/10 xfs: log NVLOOKUP xattr nvreplace operations
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
(Formerly titled "xfs: Add new name to attri/d" and described as follows:
This patch adds two new fields to the atti/d. They are nname and nnamelen. This will be used for parent pointer updates since a rename operation may cause the parent pointer to update both the name and value. So we need to carry both the new name as well as the target name in the attri/d.)
If high level code wants to do a deferred xattr nvreplace operation with the NVLOOKUP flag set, we need to push this through the log. This log item records the old name/value pair and the new name/value pair, and completely replaces one with the other. Parent pointers will need this ability to handle rename moving a child file between parents.
--- libxfs/xfs_attr.c | 20 +++++++++++++++++++- libxfs/xfs_attr.h | 4 ++-- libxfs/xfs_da_btree.h | 6 +++++- libxfs/xfs_log_format.h | 28 +++++++++++++++++++++++----- 4 files changed, 49 insertions(+), 9 deletions(-)
libxfs/xfs_attr.c +19 -1
@@ -423,6 +423,20 @@ xfs_attr_complete_op(return XFS_DAS_DONE ; args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; +if (xfs_attr_intent_op (attr) !=XFS_ATTRI_OP_FLAGS_NVREPLACE ) +return replace_state; + +/* +* NVREPLACE operations require the caller to set the old and new names +* and values explicitly. +*/ +ASSERT (args->new_namelen >0 ); + + args->name = args->new_name; + args->namelen = args->new_namelen; + args->hashval =xfs_da_hashname (args->name, args->namelen); + args->value = args->new_value; + args->valuelen = args->new_valuelen;return replace_state; }
@@ -924,9 +938,13 @@ xfs_attr_defer_replace( struct xfs_da_args *args) {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_REPLACE ;int error =0 ; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_REPLACE , &new); +if (args->op_flags &XFS_DA_OP_NVLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVREPLACE ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
libxfs/xfs_attr.h +2 -2
@@ -510,8 +510,8 @@ struct xfs_attr_intent {struct xfs_da_args *xattri_da_args;/* -* Shared buffer containing the attr name and value so that the logging -* code can share large memory buffers between log items. +* Shared buffer containing the attr name, new name, and value so that +* the logging code can share large memory buffers between log items. */ struct xfs_attri_log_nameval *xattri_nameval;
libxfs/xfs_da_btree.h +5 -1
@@ -54,11 +54,15 @@ enum xfs_dacmp {*/ typedef struct xfs_da_args { struct xfs_da_geometry *geo; /* da block geometry */ - const uint8_t *name; /* string (maybe not NULL terminated) */ + const uint8_t *name; /* string (maybe not NULL terminated) */ + const uint8_t *new_name; /* new attr name */ int namelen; /* length of string (maybe no NULL) */ + int new_namelen; /* new attr name len */ uint8_t filetype; /* filetype of inode for directories */ void *value; /* set of bytes (maybe contain NULLs) */ + void *new_value; /* new xattr value (may contain NULLs) */ int valuelen; /* length of value */ + int new_valuelen; /* length of new attr value */ unsigned int attr_filter; /* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ unsigned int attr_flags; /* XATTR_{CREATE,REPLACE} */ xfs_dahash_t hashval; /* hash value of name */
libxfs/xfs_log_format.h +23 -5
@@ -115,10 +115,11 @@ struct xfs_unmount_log_format {#define XLOG_REG_TYPE_BUD_FORMAT 26#define XLOG_REG_TYPE_ATTRI_FORMAT 27#define XLOG_REG_TYPE_ATTRD_FORMAT 28 -#define XLOG_REG_TYPE_ATTR_NAME 29 +#define XLOG_REG_TYPE_ATTR_NAME 29#define XLOG_REG_TYPE_ATTR_VALUE 30 -#define XLOG_REG_TYPE_MAX 30 - +#define XLOG_REG_TYPE_ATTR_NEWNAME 31 +#define XLOG_REG_TYPE_ATTR_NEWVALUE 32 +#define XLOG_REG_TYPE_MAX 32/* * Flags to log operation header
@@ -959,6 +960,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ #define XFS_ATTRI_OP_FLAGS_NVREMOVE 4/* Remove attr w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_NVSET 5/* Set attr with w/ vlookup */ +#define XFS_ATTRI_OP_FLAGS_NVREPLACE 6/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
@@ -976,11 +978,27 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t __pad;/* pad to 64 bit aligned */ + +/* +* For NVREPLACE, this is the length of the new xattr value. +* alfi_value_len contains the length of the old xattr value. +*/ +uint32_t alfi_new_value_len; +uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */ uint32_t alfi_op_flags;/* marks the op as a set or remove */ -uint32_t alfi_name_len;/* attr name length */ +union { +uint32_t alfi_name_len;/* attr name length */ +struct { +/* +* For NVREPLACE, these are the lengths of the old and +* new attr name. +*/ +uint16_t alfi_old_name_len; +uint16_t alfi_new_name_len; + }; + };uint32_t alfi_value_len;/* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */ };
PATCH v11 9/10 xfs_logprint: dump new attr log item fields
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Dump the new extended attribute log item fields. This was split out from the previous patch to make libxfs resyncing easier. This code needs more cleaning, which we'll do in the next few patches before moving on to the parent pointer code.
--- logprint/log_redo.c | 132 +++++++++++++++++++++++++++++++++++++++++---------- logprint/logprint.h | 6 ++ 2 files changed, 111 insertions(+), 27 deletions(-)
logprint/log_redo.c +107 -25
@@ -674,6 +674,12 @@ xfs_attri_copy_log_format(return 1 ; } +static inline unsigned int +xfs_attr_log_item_op (const struct xfs_attri_log_format *attrp) +{ +return attrp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +} + int xlog_print_trans_attri( char **ptr,
@@ -683,6 +689,10 @@ xlog_print_trans_attri(struct xfs_attri_log_format *src_f =NULL ;xlog_op_header_t *head =NULL ;uint dst_len; +unsigned int name_len =0 ; +unsigned int new_name_len =0 ; +unsigned int value_len =0 ; +unsigned int new_value_len =0 ;int error =0 ; dst_len =sizeof (struct xfs_attri_log_format );
@@ -705,27 +715,67 @@ xlog_print_trans_attri(memmove ((char *)src_f, *ptr, src_len); *ptr += src_len; -printf (_ ("ATTRI: #regs: %d name_len: %d, value_len: %d id: 0x%llx\n" ), - src_f->alfi_size, src_f->alfi_name_len, src_f->alfi_value_len, - (unsigned long long )src_f->alfi_id); +if (xfs_attr_log_item_op (src_f) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { + name_len = src_f->alfi_old_name_len; + new_name_len = src_f->alfi_new_name_len; + value_len = src_f->alfi_value_len; + new_value_len = src_f->alfi_new_value_len; + }else { + name_len = src_f->alfi_name_len; + value_len = src_f->alfi_value_len; + } + +printf (_ ("ATTRI: #regs: %d name_len: %u, new_name_len: %u, value_len: %u, new_value_len: %u id: 0x%llx\n" ), + src_f->alfi_size, + name_len, + new_name_len, + value_len, + new_value_len, + (unsigned long long )src_f->alfi_id); + +if (name_len >0 ) { +printf (_ ("\n" )); + (*i)++; + head = (xlog_op_header_t *)*ptr; +xlog_print_op_header (head, *i, ptr); + error =xlog_print_trans_attri_name (ptr, +be32_to_cpu (head->oh_len),"name" ); +if (error) + goto error; + } -if (src_f->alfi_name_len >0 ) { +if (new_name_len >0 ) {printf (_ ("\n" )); (*i)++; head = (xlog_op_header_t *)*ptr;xlog_print_op_header (head, *i, ptr); - error =xlog_print_trans_attri_name (ptr,be32_to_cpu (head->oh_len)); + error =xlog_print_trans_attri_name (ptr, +be32_to_cpu (head->oh_len),"newname" );if (error) goto error; } -if (src_f->alfi_value_len >0 ) { +if (value_len >0 ) {printf (_ ("\n" )); (*i)++; head = (xlog_op_header_t *)*ptr;xlog_print_op_header (head, *i, ptr); - error =xlog_print_trans_attri_value (ptr,be32_to_cpu (head->oh_len), - src_f->alfi_value_len); + error =xlog_print_trans_attri_value (ptr, +be32_to_cpu (head->oh_len), value_len,"value" ); +if (error) + goto error; + } + +if (new_value_len >0 ) { +printf (_ ("\n" )); + (*i)++; + head = (xlog_op_header_t *)*ptr; +xlog_print_op_header (head, *i, ptr); + error =xlog_print_trans_attri_value (ptr, +be32_to_cpu (head->oh_len), new_value_len, +"newvalue" ); +if (error) + goto error; } error:free (src_f);
@@ -736,31 +786,33 @@ xlog_print_trans_attri(int xlog_print_trans_attri_name (char **ptr, -uint src_len) +uint src_len, +const char *tag) { -printf (_ ("ATTRI: name len:%u\n" ), src_len); +printf (_ ("ATTRI: %s len:%u\n" ), tag, src_len);print_or_dump (*ptr, src_len); *ptr += src_len;return 0 ; -}/* xlog_print_trans_attri */ +}int xlog_print_trans_attri_value (char **ptr,uint src_len, -int value_len) +int value_len, +const char *tag) {int len =min (value_len, src_len); -printf (_ ("ATTRI: value len:%u\n" ), value_len); +printf (_ ("ATTRI: %s len:%u\n" ), tag, value_len);print_or_dump (*ptr, len); *ptr += src_len;return 0 ; -}/* xlog_print_trans_attri_value */ +}void xlog_recover_print_attri(
@@ -768,7 +820,10 @@ xlog_recover_print_attri( {struct xfs_attri_log_format *f, *src_f =NULL ;uint src_len, dst_len; - +unsigned int name_len =0 ; +unsigned int new_name_len =0 ; +unsigned int value_len =0 ; +unsigned int new_value_len =0 ;int region =0 ; src_f = (struct xfs_attri_log_format *)item->ri_buf[0 ].i_addr;
@@ -788,24 +843,51 @@ xlog_recover_print_attri(if (xfs_attri_copy_log_format ((char *)src_f, src_len, f)) goto out; -printf (_ ("ATTRI: #regs: %d name_len: %d, value_len: %d id: 0x%llx\n" ), - f->alfi_size, f->alfi_name_len, f->alfi_value_len, (unsigned long long )f->alfi_id); +if (xfs_attr_log_item_op (f) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { + name_len = f->alfi_old_name_len; + new_name_len = f->alfi_new_name_len; + value_len = f->alfi_value_len; + new_value_len = f->alfi_new_value_len; + }else { + name_len = f->alfi_name_len; + value_len = f->alfi_value_len; + } + +printf (_ ("ATTRI: #regs: %d name_len: %u, new_name_len: %u, value_len: %d, new_value_len: %u id: 0x%llx\n" ), + f->alfi_size, + name_len, + new_name_len, + value_len, + new_value_len, + (unsigned long long )f->alfi_id); -if (f->alfi_name_len >0 ) { +if (name_len >0 ) { region++; -printf (_ ("ATTRI: name len:%u\n" ), f->alfi_name_len); +printf (_ ("ATTRI: name len:%u\n" ), name_len);print_or_dump ((char *)item->ri_buf[region].i_addr, - f->alfi_name_len); + name_len); } -if (f->alfi_value_len >0 ) { -int len = f->alfi_value_len; +if (new_name_len >0 ) { + region++; +printf (_ ("ATTRI: newname len:%u\n" ), new_name_len); +print_or_dump ((char *)item->ri_buf[region].i_addr, + new_name_len); + } + +if (value_len >0 ) { +int len =min (MAX_ATTR_VAL_PRINT , value_len); + + region++; +printf (_ ("ATTRI: value len:%u\n" ), value_len); +print_or_dump ((char *)item->ri_buf[region].i_addr, len); + } -if (len >MAX_ATTR_VAL_PRINT ) - len =MAX_ATTR_VAL_PRINT ; +if (new_value_len >0 ) { +int len =min (MAX_ATTR_VAL_PRINT , new_value_len); region++; -printf (_ ("ATTRI: value len:%u\n" ), f->alfi_value_len); +printf (_ ("ATTRI: newvalue len:%u\n" ), new_value_len);print_or_dump ((char *)item->ri_buf[region].i_addr, len); }
logprint/logprint.h +4 -2
@@ -59,8 +59,10 @@ extern void xlog_recover_print_bud(struct xlog_recover_item *item);#define MAX_ATTR_VAL_PRINT 128extern int xlog_print_trans_attri (char **ptr,uint src_len,int *i); -extern int xlog_print_trans_attri_name (char **ptr,uint src_len); -extern int xlog_print_trans_attri_value (char **ptr,uint src_len,int value_len); +extern int xlog_print_trans_attri_name (char **ptr,uint src_len, +const char *tag); +extern int xlog_print_trans_attri_value (char **ptr,uint src_len,int value_len, +const char *tag);extern void xlog_recover_print_attri (struct xlog_recover_item *item);extern int xlog_print_trans_attrd (char **ptr,uint len);extern void xlog_recover_print_attrd (struct xlog_recover_item *item);
PATCH v11 10/10 xfs_logprint: print missing attri header fields
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Not sure why logprint doesn't print the op flags, inode, or attr filter fields. Make it do that.
--- logprint/log_redo.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
logprint/log_redo.c +8 -2
@@ -725,8 +725,11 @@ xlog_print_trans_attri( value_len = src_f->alfi_value_len; } - printf(_("ATTRI: #regs: %d name_len: %u, new_name_len: %u, value_len: %u, new_value_len: %u id: 0x%llx\n"), + printf(_("ATTRI: #regs: %d f: 0x%x, ino: 0x%llx, attr_filter: 0x%x, name_len: %u, new_name_len: %u, value_len: %u, new_value_len: %u id: 0x%llx\n"), src_f->alfi_size, + src_f->alfi_op_flags, + (unsigned long long)src_f->alfi_ino, + src_f->alfi_attr_filter, name_len, new_name_len, value_len,
@@ -853,8 +856,11 @@ xlog_recover_print_attri( value_len = f->alfi_value_len; } - printf(_("ATTRI: #regs: %d name_len: %u, new_name_len: %u, value_len: %d, new_value_len: %u id: 0x%llx\n"), + printf(_("ATTRI: #regs: %d f: 0x%x, ino: 0x%llx, attr_filter: 0x%x, name_len: %u, new_name_len: %u, value_len: %u, new_value_len: %u id: 0x%llx\n"), f->alfi_size, + f->alfi_op_flags, + (unsigned long long)f->alfi_ino, + f->alfi_attr_filter, name_len, new_name_len, value_len,
v11 cover xfsprogs: Parent Pointers
In reply to Darrick J. WongHi all,
NOTE: Darrick has tweaked some of these patches to match the kernel code.
This is the latest parent pointer attributes for xfs.
The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point.
This set can be viewed on github here
https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv10
And the corresponding xfsprogs code is here
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v10
This set has been tested with the below parent pointers tests
https://lore.kernel.org/fstests/[email protected]/T/#t
Updates since v9:
Reordered patches 2 and 3 to be 6 and 7
xfs: Add xfs_verify_pptr
moved parent pointer validators to xfs_parent
xfs: Add parent pointer ioctl
Extra validation checks for fs id
added missing release for the inode
use GFP_KERNEL flags for malloc/realloc reworked ioctl to use pptr listenty and flex array
NEW
xfs: don't remove the attr fork when parent pointers are enabled
NEW
directory lookups should return diroffsets too
NEW
xfs: move/add parent pointer validators to xfs_parent
Updates since v8:
xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec
xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas
xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates
xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free
xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once Fix xfs_rename to disallow reservationless quotas Fix double unlock on dqattach fail
Move parent frees to out_release_wip
xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename
Questions comments and feedback appreciated!
Thanks all!
Allison
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
--- db/Makefile | 2 db/attr.c | 65 +++++ db/attrshort.c | 51 ++++ db/hash.c | 374 +++++++++++++++++++++++++++ db/metadump.c | 646 +++++++++++++++++++++------------------------- db/namei.c | 335 ++++++++++++++++++++++++ db/obfuscate.c | 348 +++++++++++++++++++++++++ db/obfuscate.h | 16 + include/handle.h | 1 include/libxfs.h | 2 include/xfs.h | 1 include/xfs_inode.h | 6 io/parent.c | 527 +++++++++++++------------------------- libfrog/Makefile | 2 libfrog/fsgeom.c | 4 libfrog/getparents.c | 338 ++++++++++++++++++++++++ libfrog/getparents.h | 36 +++ libfrog/paths.c | 167 ++++++++++++ libfrog/paths.h | 25 ++ libhandle/handle.c | 7 libxfs/Makefile | 3 libxfs/init.c | 7 libxfs/libxfs_api_defs.h | 6 libxfs/libxfs_priv.h | 6 libxfs/util.c | 14 + libxfs/xfs_attr.c | 21 + libxfs/xfs_attr.h | 14 + libxfs/xfs_attr_leaf.c | 6 libxfs/xfs_attr_sf.h | 1 libxfs/xfs_da_format.h | 34 ++ libxfs/xfs_defer.c | 28 ++ libxfs/xfs_defer.h | 8 - libxfs/xfs_format.h | 4 libxfs/xfs_fs.h | 2 libxfs/xfs_fs_staging.h | 84 ++++++ libxfs/xfs_log_format.h | 1 libxfs/xfs_log_rlimit.c | 53 ++++ libxfs/xfs_parent.c | 318 +++++++++++++++++++++++ libxfs/xfs_parent.h | 104 +++++++ libxfs/xfs_sb.c | 4 libxfs/xfs_trans_resv.c | 324 +++++++++++++++++++---- libxfs/xfs_trans_space.h | 8 - logprint/log_redo.c | 81 ++++++ man/man3/xfsctl.3 | 63 ++++ man/man8/xfs_db.8 | 40 +++ man/man8/xfs_io.8 | 30 +- mkfs/proto.c | 48 +++ mkfs/xfs_mkfs.c | 31 ++ repair/attr_repair.c | 19 + repair/phase6.c | 6 scrub/common.c | 41 +++ 51 files changed, 3562 insertions(+), 800 deletions(-) create mode 100644 db/obfuscate.c create mode 100644 db/obfuscate.h create mode 100644 libfrog/getparents.c create mode 100644 libfrog/getparents.h create mode 100644 libxfs/xfs_fs_staging.h create mode 100644 libxfs/xfs_parent.c create mode 100644 libxfs/xfs_parent.h
PATCH v11 1/32 xfsprogs: Increase XFS_DEFER_OPS_NR_INODES to 5
From: Allison Henderson <[email protected]>
Source kernel commit: e9dc6a1e293b7e3843cd3868603801a1af2704c3
Renames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.
--- libxfs/libxfs_priv.h | 1 + libxfs/xfs_defer.c | 28 ++++++++++++++++++++++++++-- libxfs/xfs_defer.h | 8 +++++++- 3 files changed, 34 insertions(+), 3 deletions(-)
libxfs/libxfs_priv.h +1 -0
@@ -478,6 +478,7 @@ void __xfs_buf_mark_corrupt(struct xfs_buf *bp, xfs_failaddr_t fa); __mode = __mode;/* no set-but-unused warning */ \ })#define xfs_lock_two_inodes (ip0,mode0,ip1,mode1) ((void) 0) +#define xfs_lock_inodes (ips,num_ips,mode) ((void) 0)/* space allocation */ #define XFS_EXTENT_BUSY_DISCARDED 0x01/* undergoing a discard op. */
libxfs/xfs_defer.c +26 -2
@@ -815,13 +815,37 @@ xfs_defer_ops_continue(struct xfs_trans *tp, struct xfs_defer_resources *dres) { - unsigned int i; + unsigned int i, j; +struct xfs_inode *sips[XFS_DEFER_OPS_NR_INODES ]; +struct xfs_inode *temp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );ASSERT (!(tp->t_flags &XFS_TRANS_DIRTY ));/* Lock the captured resources to the new transaction. */ -if (dfc->dfc_held.dr_inos ==2 ) +if (dfc->dfc_held.dr_inos >2 ) { +/* +* Renames with parent pointer updates can lock up to 5 inodes, +* sorted by their inode number. So we need to make sure they +* are relocked in the same way. +*/ +memset (sips,0 ,sizeof (sips)); +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) + sips[i] = dfc->dfc_held.dr_ip[i]; + +/* Bubble sort of at most 5 inodes */ +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) { +for (j =1 ; j < dfc->dfc_held.dr_inos; j++) { +if (sips[j]->i_ino < sips[j-1 ]->i_ino) { + temp = sips[j]; + sips[j] = sips[j-1 ]; + sips[j-1 ] = temp; + } + } + } + +xfs_lock_inodes (sips, dfc->dfc_held.dr_inos,XFS_ILOCK_EXCL ); + }else if (dfc->dfc_held.dr_inos ==2 )xfs_lock_two_inodes (dfc->dfc_held.dr_ip[0 ],XFS_ILOCK_EXCL , dfc->dfc_held.dr_ip[1 ],XFS_ILOCK_EXCL );else if (dfc->dfc_held.dr_inos ==1 )
libxfs/xfs_defer.h +7 -1
@@ -70,7 +70,13 @@ extern const struct xfs_defer_op_type xfs_attr_defer_type;/* * Deferred operation item relogging limits. */ -#define XFS_DEFER_OPS_NR_INODES 2/* join up to two inodes */ + +/* +* Rename w/ parent pointers can require up to 5 inodes with deferred ops to +* be joined to the transaction: src_dp, target_dp, src_ip, target_ip, and wip. +* These inodes are locked in sorted order by their inode numbers +*/ +#define XFS_DEFER_OPS_NR_INODES 5#define XFS_DEFER_OPS_NR_BUFS 2/* join up to two buffers */ /* Resources that must be held across a transaction roll. */
PATCH v11 2/32 xfs: add parent pointer support to attribute code
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer entries; it uses reserved blocks like XFS_ATTR_ROOT.
--- libxfs/xfs_attr.c | 3 ++- libxfs/xfs_da_format.h | 5 ++++- libxfs/xfs_log_format.h | 1 + 3 files changed, 7 insertions(+), 2 deletions(-)
libxfs/xfs_attr.c +2 -1
@@ -990,7 +990,8 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; -bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool rsvd = (args->attr_filter & (XFS_ATTR_ROOT | +XFS_ATTR_PARENT ));bool is_remove = args->op_flags &XFS_DA_OP_REMOVE ;int error, local;int rmt_blks =0 ;
libxfs/xfs_da_format.h +4 -1
@@ -688,12 +688,15 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */ #define XFS_ATTR_SECURE_BIT 2/* limit access to secure attrs */ +#define XFS_ATTR_PARENT_BIT 3/* parent pointer attrs */ #define XFS_ATTR_INCOMPLETE_BIT 7/* attr in middle of create/delete */ #define XFS_ATTR_LOCAL (1u << XFS_ATTR_LOCAL_BIT)#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT) +#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_NSP_ONDISK_MASK \ + (XFS_ATTR_ROOT | XFS_ATTR_SECURE | XFS_ATTR_PARENT)/* * Alignment for namelist and valuelist entries (since they are mixed
libxfs/xfs_log_format.h +1 -0
@@ -969,6 +969,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/*
PATCH v11 3/32 xfs: define parent pointer ondisk extended attribute format
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to define the parent pointer attribute format before we start adding support for it into all the code that needs to use it. The EA format we will use encodes the following information:
name={parent inode #, parent inode generation, dirent namehash}
value={dirent name}The inode/gen gives all the information we need to reliably identify the parent without requiring child->parent lock ordering, and allows userspace to do pathname component level reconstruction without the kernel ever needing to verify the parent itself as part of ioctl calls. Storing the dirent name hash in the key reduces hash collisions if a file is hardlinked multiple times in the same directory.
By using the NVLOOKUP mode in the extended attribute code to match parent pointers using both the xattr name and value, we can identify the exact parent pointer EA we need to modify/remove in rename/unlink operations without searching the entire EA space.
By storing the dirent name, we have enough information to be able to validate and reconstruct damaged directory trees. Earlier iterations of this patchset encoded the directory offset in the parent pointer key, but this format required repair to keep that in sync across directory rebuilds, which is unnecessary complexity.
--- libxfs/xfs_da_format.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
libxfs/xfs_da_format.h +21 -0
@@ -805,4 +805,25 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); +/* +* Parent pointer attribute format definition +* +* The xattr name encodes the parent inode number, generation and the crc32c +* hash of the dirent name. +* +* The xattr value contains the dirent name. +*/ +struct xfs_parent_name_rec { +__be64 p_ino; +__be32 p_gen; +__be32 p_namehash; +}; + +/* +* Maximum size of the dirent name that can be stored in a parent pointer. +* This matches the maximum dirent name length. +*/ +#define XFS_PARENT_DIRENT_NAME_MAX_SIZE \ + (MAXNAMELEN - 1) +#endif /* __XFS_DA_FORMAT_H__ */
PATCH v11 4/32 xfs: add parent pointer validator functions
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings. So we need to modify attr_namecheck to verify parent pointer records when the XFS_ATTR_PARENT flag is set. At the same time, we need to validate attr values during log recovery if the xattr is really a parent pointer.
--- libxfs/Makefile | 2 + libxfs/xfs_attr.c | 10 ++++- libxfs/xfs_attr.h | 3 +- libxfs/xfs_da_format.h | 8 ++++ libxfs/xfs_parent.c | 92 ++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_parent.h | 16 ++++++++ repair/attr_repair.c | 19 ++++++---- 7 files changed, 140 insertions(+), 10 deletions(-) create mode 100644 libxfs/xfs_parent.c create mode 100644 libxfs/xfs_parent.h
libxfs/Makefile +2 -0
@@ -45,6 +45,7 @@ HFILES = \ xfs_ialloc_btree.h \ xfs_inode_buf.h \ xfs_inode_fork.h \ + xfs_parent.h \ xfs_quota_defs.h \ xfs_refcount.h \ xfs_refcount_btree.h \
@@ -92,6 +93,7 @@ CFILES = cache.c \ xfs_inode_fork.c \ xfs_ialloc_btree.c \ xfs_log_rlimit.c \ + xfs_parent.c \ xfs_refcount.c \ xfs_refcount_btree.c \ xfs_rmap.c \
libxfs/xfs_attr.c +8 -2
@@ -24,6 +24,7 @@#include "xfs_quota_defs.h" #include "xfs_trans_space.h" #include "xfs_trace.h" +#include "xfs_parent.h" struct kmem_cache *xfs_attr_intent_cache;
@@ -1594,9 +1595,14 @@ xfs_attr_node_get(/* Returns true if the attribute entry name is valid. */ bool xfs_attr_namecheck ( -const void *name, -size_t length) +struct xfs_mount *mp, +const void *name, +size_t length, +unsigned int flags) { +if (flags &XFS_ATTR_PARENT ) +return xfs_parent_namecheck (mp, name, length, flags); +/* * MAXNAMELEN includes the trailing null, but (name/length) leave it * out, so use >= for the length check.
libxfs/xfs_attr.h +2 -1
@@ -552,7 +552,8 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, +unsigned int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);
libxfs/xfs_da_format.h +8 -0
@@ -731,6 +731,14 @@ xfs_attr3_leaf_name(xfs_attr_leafblock_t *leafp, int idx)return &((char *)leafp)[be16_to_cpu (entries[idx].nameidx)]; } +static inline int +xfs_attr3_leaf_flags (xfs_attr_leafblock_t *leafp,int idx) +{ +struct xfs_attr_leaf_entry *entries =xfs_attr3_leaf_entryp (leafp); + +return entries[idx].flags; +} +static inline xfs_attr_leaf_name_remote_t *xfs_attr3_leaf_name_remote (xfs_attr_leafblock_t *leafp,int idx) {
libxfs/xfs_parent.c new file +92 -0
92 lines not on this page · raw message
libxfs/xfs_parent.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022-2023 Oracle, Inc. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* Metadata validators */ +bool xfs_parent_namecheck (struct xfs_mount *mp, +const struct xfs_parent_name_rec *rec,size_t reclen, +unsigned int attr_flags); +bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value, +size_t valuelen); + +#endif /* __XFS_PARENT_H__ */
repair/attr_repair.c +12 -7
@@ -293,8 +293,9 @@ process_shortform_attr( }/* namecheck checks for null chars in attr names. */ -if (!libxfs_attr_namecheck (currententry->nameval, - currententry->namelen)) { +if (!libxfs_attr_namecheck (mp, currententry->nameval, + currententry->namelen, + currententry->flags)) {do_warn (_ ("entry contains illegal character in shortform attribute name\n" )); junkit =1 ;
@@ -454,12 +455,14 @@ process_leaf_attr_local( xfs_dablk_t da_bno, xfs_ino_t ino) { -xfs_attr_leaf_name_local_t *local; +xfs_attr_leaf_name_local_t *local; +int flags; local =xfs_attr3_leaf_name_local (leaf, i); + flags =xfs_attr3_leaf_flags (leaf, i);if (local->namelen ==0 || - !libxfs_attr_namecheck (local->nameval, - local->namelen)) { + !libxfs_attr_namecheck (mp, local->nameval, + local->namelen, flags)) {do_warn (_ ("attribute entry %d in attr block %u, inode %" PRIu64" has bad name (namelen = %d)\n" ), i, da_bno, ino, local->namelen);
@@ -510,12 +513,14 @@ process_leaf_attr_remote( {xfs_attr_leaf_name_remote_t *remotep;char * value; +int flags; remotep =xfs_attr3_leaf_name_remote (leaf, i); + flags =xfs_attr3_leaf_flags (leaf, i); if (remotep->namelen == 0 || - !libxfs_attr_namecheck(remotep->name, - remotep->namelen) || + !libxfs_attr_namecheck(mp, remotep->name, + remotep->namelen, flags) || be32_to_cpu(entry->hashval) != libxfs_da_hashname((unsigned char *)&remotep->name[0], remotep->namelen) ||
PATCH v11 5/32 xfs: extend transaction reservations for parent attributes
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.
--- libxfs/libxfs_priv.h | 1 libxfs/xfs_trans_resv.c | 324 +++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 273 insertions(+), 52 deletions(-)
libxfs/libxfs_priv.h +1 -0
@@ -523,6 +523,7 @@ static inline int retzero(void) { return 0; }#define xfs_icreate_log (tp, agno, agbno, cnt, isize, len, gen) ((void) 0)#define xfs_sb_validate_fsb_count (sbp, nblks) (0) +#define xlog_calc_iovec_len (len) roundup(len, sizeof(uint32_t))/* * Prototypes for kernel static functions that are aren't in their
libxfs/xfs_trans_resv.c +272 -52
418 lines not on this page · raw message
PATCH v11 6/32 xfs: parent pointer attribute creation
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add parent pointer attribute during xfs_create, and subroutines to initialize attributes
--- include/libxfs.h | 1 include/xfs_inode.h | 6 +++ libxfs/init.c | 3 + libxfs/libxfs_priv.h | 3 + libxfs/xfs_attr.c | 4 +- libxfs/xfs_attr.h | 4 +- libxfs/xfs_parent.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_parent.h | 44 +++++++++++++++++++ 8 files changed, 176 insertions(+), 5 deletions(-)
include/libxfs.h +1 -0
@@ -78,6 +78,7 @@ struct iomap;#include "xfs_refcount_btree.h" #include "xfs_refcount.h" #include "xfs_btree_staging.h" +#include "xfs_parent.h" #ifndef ARRAY_SIZE #define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0]))
include/xfs_inode.h +6 -0
@@ -175,6 +175,12 @@ static inline struct inode *VFS_I(struct xfs_inode *ip)return &ip->i_vnode; } +/* convert from const xfs inode to const vfs inode */ +static inline const struct inode *VFS_IC (const struct xfs_inode *ip) +{ +return &ip->i_vnode; +} +/* We only have i_size in the xfs inode in userspace */ static inline loff_t i_size_read (struct inode *inode) {
libxfs/init.c +3 -0
@@ -258,6 +258,8 @@ init_caches(void) "xfs_extfree_item"); xfs_trans_cache =kmem_cache_init (sizeof (struct xfs_trans ),"xfs_trans" ); + xfs_parent_intent_cache =kmem_cache_init ( +sizeof (struct xfs_parent_defer ),"xfs_parent_defer" ); } static int
@@ -275,6 +277,7 @@ destroy_caches(void)xfs_btree_destroy_cur_caches (); leaked +=kmem_cache_destroy (xfs_extfree_item_cache); leaked +=kmem_cache_destroy (xfs_trans_cache); + leaked +=kmem_cache_destroy (xfs_parent_intent_cache);return leaked; }
libxfs/libxfs_priv.h +2 -1
@@ -616,7 +616,8 @@ int libxfs_zero_extent(struct xfs_inode *ip, xfs_fsblock_t start_fsb,/* xfs_log.c */ bool xfs_log_check_lsn (struct xfs_mount *,xfs_lsn_t );void xfs_log_item_init (struct xfs_mount *,struct xfs_log_item *,int ); -#define xfs_attr_use_log_assist (mp) (0) +#define xfs_attr_grab_log_assist (mp) (0) +#define xfs_attr_rele_log_assist (mp) ((void) 0)#define xlog_drop_incompat_feat (log) do { } while (0)#define xfs_log_in_recovery (mp) (false)
libxfs/xfs_attr.c +2 -2
@@ -893,7 +893,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set or remove) */
@@ -911,7 +911,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {
libxfs/xfs_attr.h +3 -1
@@ -549,6 +549,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
@@ -557,7 +558,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/* * Check to see if the attr should be upgraded from non-existent or shortform to * single-leaf-block attribute list.
libxfs/xfs_parent.c +116 -0
131 lines not on this page · raw message
libxfs/xfs_parent.h +43 -1
47 lines not on this page · raw message
PATCH v11 7/32 xfs: add parent attributes to link
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_link to add a parent pointer to the inode.
--- libxfs/xfs_trans_space.h | 2 -- 1 file changed, 2 deletions(-)
libxfs/xfs_trans_space.h +0 -2
@@ -84,8 +84,6 @@ (2 * (mp)->m_alloc_maxlevels)#define XFS_GROWFSRT_SPACE_RES (mp,b) \ ((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK)) -#define XFS_LINK_SPACE_RES (mp,nl) \ - XFS_DIRENTER_SPACE_RES(mp,nl)#define XFS_MKDIR_SPACE_RES (mp,nl) \ (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_QM_DQALLOC_SPACE_RES (mp) \
PATCH v11 8/32 xfs: add parent attributes to symlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_symlink to add a parent pointer to the inode.
--- libxfs/xfs_trans_space.h | 2 -- 1 file changed, 2 deletions(-)
libxfs/xfs_trans_space.h +0 -2
@@ -95,8 +95,6 @@ XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl)) -#define XFS_SYMLINK_SPACE_RES (mp,nl,b) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl) + (b))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
PATCH v11 9/32 xfs: remove parent pointers in unlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the parent pointer attribute during unlink
--- libxfs/xfs_attr.c | 2 +- libxfs/xfs_attr.h | 1 + libxfs/xfs_parent.c | 22 ++++++++++++++++++++++ libxfs/xfs_parent.h | 4 ++++ libxfs/xfs_trans_space.h | 2 -- repair/phase6.c | 6 +++--- 6 files changed, 31 insertions(+), 6 deletions(-)
libxfs/xfs_attr.c +1 -1
@@ -957,7 +957,7 @@ xfs_attr_defer_replace( }/* Removes an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_remove (struct xfs_da_args *args) {
libxfs/xfs_attr.h +1 -0
@@ -550,6 +550,7 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args); +int xfs_attr_defer_remove (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
libxfs/xfs_parent.c +22 -0
@@ -183,6 +183,28 @@ xfs_parent_add(return xfs_attr_defer_add (args); } +/* Remove a parent pointer to reflect a dirent removal. */ +int +xfs_parent_remove ( +struct xfs_trans *tp, +struct xfs_parent_defer *parent, +struct xfs_inode *dp, +const struct xfs_name *parent_name, +struct xfs_inode *child) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, dp, parent_name, child); + args->hashval =xfs_parent_hashname (dp, parent); + + args->trans = tp; + args->dp = child; + +xfs_init_parent_davalue (&parent->args, parent_name); + +return xfs_attr_defer_remove (args); +} +/* Cancel a parent pointer operation. */ void __xfs_parent_cancel(
libxfs/xfs_parent.h +4 -0
@@ -41,6 +41,10 @@ xfs_parent_start(int xfs_parent_add (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp,const struct xfs_name *parent_name,struct xfs_inode *child); +int xfs_parent_remove (struct xfs_trans *tp,struct xfs_parent_defer *parent, +struct xfs_inode *dp,const struct xfs_name *parent_name, +struct xfs_inode *child); +void __xfs_parent_cancel (struct xfs_mount *mp,struct xfs_parent_defer *parent); static inline void
libxfs/xfs_trans_space.h +0 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_REMOVE_SPACE_RES (mp) \ - XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \
repair/phase6.c +3 -3
@@ -1266,7 +1266,7 @@ longform_dir2_rebuild( libxfs_dir_ino_validate(mp, pip.i_ino)) pip.i_ino = mp->m_sb.sb_rootino; - nres =XFS_REMOVE_SPACE_RES (mp); + nres =XFS_DIRREMOVE_SPACE_RES (mp); error = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_remove, nres,0 ,0 , &tp);if (error)res_failed (error);
@@ -1371,7 +1371,7 @@ dir2_kill_block(int nres;xfs_trans_t *tp; - nres =XFS_REMOVE_SPACE_RES (mp); + nres =XFS_DIRREMOVE_SPACE_RES (mp); error = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_remove, nres,0 ,0 , &tp);if (error)res_failed (error);
@@ -2887,7 +2887,7 @@ process_dir_inode(* inode but it's easier than wedging a * new define in ourselves. */ - nres = no_modify ?0 :XFS_REMOVE_SPACE_RES (mp); + nres = no_modify ?0 :XFS_DIRREMOVE_SPACE_RES (mp); error = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_remove, nres,0 ,0 , &tp);if (error)
PATCH v11 10/32 xfs: Add parent pointers to rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer.
--- libxfs/xfs_attr.c | 2 +- libxfs/xfs_attr.h | 1 + libxfs/xfs_parent.c | 58 ++++++++++++++++++++++++++++++++++++++++++---- libxfs/xfs_parent.h | 23 +++++++++++++++++- libxfs/xfs_trans_space.h | 2 -- 5 files changed, 76 insertions(+), 10 deletions(-)
libxfs/xfs_attr.c +1 -1
@@ -934,7 +934,7 @@ xfs_attr_defer_add( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_replace (struct xfs_da_args *args) {
libxfs/xfs_attr.h +1 -0
@@ -551,6 +551,7 @@ int xfs_attr_get_ilocked(struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_defer_remove (struct xfs_da_args *args); +int xfs_attr_defer_replace (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
libxfs/xfs_parent.c +53 -5
94 lines not on this page · raw message
libxfs/xfs_parent.h +21 -2
@@ -21,10 +21,13 @@ extern struct kmem_cache *xfs_parent_intent_cache;*/ struct xfs_parent_defer {struct xfs_parent_name_rec rec; +struct xfs_parent_name_rec new_rec;struct xfs_da_args args; +bool have_log; }; -int __xfs_parent_init (struct xfs_mount *mp,struct xfs_parent_defer **parentp); +int __xfs_parent_init (struct xfs_mount *mp,bool grab_log, +struct xfs_parent_defer **parentp);static inline int xfs_parent_start(
@@ -34,7 +37,19 @@ xfs_parent_start( *pp =NULL ;if (xfs_has_parent (mp)) -return __xfs_parent_init (mp, pp); +return __xfs_parent_init (mp, true, pp); +return 0 ; +} + +static inline int +xfs_parent_start_locked ( +struct xfs_mount *mp, +struct xfs_parent_defer **pp) +{ + *pp =NULL ; + +if (xfs_has_parent (mp)) +return __xfs_parent_init (mp, false, pp);return 0 ; }
@@ -44,6 +59,10 @@ int xfs_parent_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,int xfs_parent_remove (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp,const struct xfs_name *parent_name,struct xfs_inode *child); +int xfs_parent_replace (struct xfs_trans *tp,struct xfs_parent_defer *parent, +struct xfs_inode *old_dp,const struct xfs_name *old_name, +struct xfs_inode *new_dp,const struct xfs_name *new_name, +struct xfs_inode *child);void __xfs_parent_cancel (struct xfs_mount *mp,struct xfs_parent_defer *parent);
libxfs/xfs_trans_space.h +0 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_RENAME_SPACE_RES (mp,nl) \ - (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
PATCH v11 11/32 xfsprogs: pass the attr value to put_listent when possible
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Source kernel commit: 45a4366e940b853beb4d65efec08e204aee0113d
Pass the attr value to put_listent when we have local xattrs or shortform xattrs.
--- libxfs/xfs_attr.h | 5 +++-- libxfs/xfs_attr_sf.h | 1 + 2 files changed, 4 insertions(+), 2 deletions(-)
libxfs/xfs_attr.h +3 -2
@@ -47,8 +47,9 @@ struct xfs_attrlist_cursor_kern {/* void; state communicated via *context */ -typedef void (*put_listent_func_t )(struct xfs_attr_list_context *,int , -unsigned char *,int ,int ); +typedef void (*put_listent_func_t )(struct xfs_attr_list_context *context, +int flags,unsigned char *name,int namelen,void *value, +int valuelen);struct xfs_attr_list_context {struct xfs_trans *tp;
libxfs/xfs_attr_sf.h +1 -0
@@ -24,6 +24,7 @@ typedef struct xfs_attr_sf_sort {uint8_t flags;/* flags bits (see xfs_attr_leaf.h) */ xfs_dahash_t hash;/* this entry's hash value */ unsigned char *name;/* name value, pointer into buffer */ +void *value; } xfs_attr_sf_sort_t;#define XFS_ATTR_SF_ENTSIZE_MAX /* max space for name&value */ \
PATCH v11 12/32 xfs: add a libxfs header file for staging new ioctls
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new xfs_fs_staging.h header where we can land experimental ioctls without committing them to any stable interfaces anywhere.
--- include/libxfs.h | 1 + include/xfs.h | 1 + libxfs/Makefile | 1 + libxfs/libxfs_priv.h | 1 + libxfs/xfs_fs_staging.h | 18 ++++++++++++++++++ 5 files changed, 22 insertions(+) create mode 100644 libxfs/xfs_fs_staging.h
include/libxfs.h +1 -0
@@ -23,6 +23,7 @@#include "xfs_types.h" #include "xfs_fs.h" +#include "xfs_fs_staging.h" #include "xfs_arch.h" #include "xfs_shared.h"
include/xfs.h +1 -0
@@ -44,5 +44,6 @@ extern int xfs_assert_largefile[sizeof(off_t)-8];/* Include deprecated/compat pre-vfs xfs-specific symbols */ #include <xfs/xfs_fs_compat.h> #include <xfs/xfs_fs.h> +#include <xfs/xfs_fs_staging.h> #endif /* __XFS_H__ */
libxfs/Makefile +1 -0
@@ -14,6 +14,7 @@ LTLDFLAGS += -static # headers to install in include/xfs PKGHFILES = xfs_fs.h \ + xfs_fs_staging.h \ xfs_types.h \ xfs_da_format.h \ xfs_format.h \
libxfs/libxfs_priv.h +1 -0
@@ -56,6 +56,7 @@#include "xfs_arch.h" #include "xfs_fs.h" +#include "xfs_fs_staging.h" #include "libfrog/crc32c.h" #include <sys/xattr.h>
libxfs/xfs_fs_staging.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: LGPL-2.1 */ +/* +* Copyright (C) 2020-2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_FS_STAGING_H__ +#define __XFS_FS_STAGING_H__ + +/* +* Experimental system calls, ioctls and data structures supporting them. +* Nothing in here should be considered part of a stable interface of any kind. +* +* If you add an ioctl here, please leave a comment in xfs_fs.h marking it +* reserved. If you promote anything out of this file, please leave a comment +* explaining where it went. +*/ + +#endif /* __XFS_FS_STAGING_H__ */
PATCH v11 13/32 xfs: Add parent pointer ioctl
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch adds a new file ioctl to retrieve the parent pointer of a given inode
--- libxfs/xfs_fs.h | 1 + libxfs/xfs_fs_staging.h | 66 +++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_parent.c | 40 ++++++++++++++++++++++++++++ libxfs/xfs_parent.h | 23 ++++++++++++++++ man/man3/xfsctl.3 | 63 +++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 193 insertions(+)
libxfs/xfs_fs.h +1 -0
@@ -796,6 +796,7 @@ struct xfs_scrub_metadata {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +/* XFS_IOC_GETPARENTS ---- staging 62 */ /* * ioctl commands that replace IRIX syssgi()'s
libxfs/xfs_fs_staging.h +66 -0
70 lines not on this page · raw message
libxfs/xfs_parent.c +40 -0
@@ -276,3 +276,43 @@ xfs_parent_calc_space_res(return XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ) +XFS_NEXTENTADD_SPACE_RES (mp, namelen,XFS_ATTR_FORK ); } + +/* Convert an ondisk parent pointer to the incore format. */ +void +xfs_parent_irec_from_disk ( +struct xfs_parent_name_irec *irec, +const struct xfs_parent_name_rec *rec, +const void *value, +unsigned int valuelen) +{ + irec->p_ino =be64_to_cpu (rec->p_ino); + irec->p_gen =be32_to_cpu (rec->p_gen); + irec->p_namehash =be32_to_cpu (rec->p_namehash); + irec->p_namelen = valuelen; +memcpy (irec->p_name, value, valuelen); +} + +/* Convert an incore parent pointer to the ondisk attr name format. */ +void +xfs_parent_irec_to_disk ( +struct xfs_parent_name_rec *rec, +const struct xfs_parent_name_irec *irec) +{ + rec->p_ino =cpu_to_be64 (irec->p_ino); + rec->p_gen =cpu_to_be32 (irec->p_gen); + rec->p_namehash =cpu_to_be32 (irec->p_namehash); +} + +/* Compute p_namehash for the this parent pointer. */ +void +xfs_parent_irec_hashname ( +struct xfs_mount *mp, +struct xfs_parent_name_irec *irec) +{ +struct xfs_name dname = { + .name = irec->p_name, + .len = irec->p_namelen, + }; + + irec->p_namehash =xfs_dir2_hashname (mp, &dname); +}
libxfs/xfs_parent.h +23 -0
@@ -78,4 +78,27 @@ xfs_parent_finish(unsigned int xfs_parent_calc_space_res (struct xfs_mount *mp,unsigned int namelen); +/* +* Incore version of a parent pointer, also contains dirent name so callers +* can pass/obtain all the parent pointer information in a single structure +*/ +struct xfs_parent_name_irec { +/* Parent pointer attribute name fields */ +xfs_ino_t p_ino; +uint32_t p_gen; +xfs_dahash_t p_namehash; + +/* Parent pointer attribute value fields */ +uint8_t p_namelen; +unsigned char p_name[MAXNAMELEN ]; +}; + +void xfs_parent_irec_from_disk (struct xfs_parent_name_irec *irec, +const struct xfs_parent_name_rec *rec,const void *value, +unsigned int valuelen); +void xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec, +const struct xfs_parent_name_irec *irec); +void xfs_parent_irec_hashname (struct xfs_mount *mp, +struct xfs_parent_name_irec *irec); +#endif /* __XFS_PARENT_H__ */
man/man3/xfsctl.3 +63 -0
69 lines not on this page · raw message
PATCH v11 14/32 xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Dave and I were discussing some recent test regressions as a result of me turning on nrext64=1 on realtime filesystems, when we noticed that the minimum log size of a 32M filesystem jumped from 954 blocks to 4287 blocks.
Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size contains the maximum estimated amount of space needed for a local format xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES, which requires units of blocks. This has resulted in an overestimation of the minimum log size over the years.
We should nominally correct this, but there's a backwards compatibility problem -- if we enable it now, the minimum log size will decrease. If a corrected mkfs formats a filesystem with this new smaller log size, a user will encounter mount failures on an uncorrected kernel due to the larger minimum log size computations there.
However, the large extent counters feature is still EXPERIMENTAL, so we can gate the correction on that feature (or any features that get added after that) being enabled. Any filesystem with nrext64 or any of the as-yet-undefined feature bits turned on will be rejected by old uncorrected kernels, so this should be safe even in the upgrade case.
--- libxfs/xfs_log_rlimit.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
libxfs/xfs_log_rlimit.c +43 -0
@@ -16,6 +16,39 @@#include "xfs_bmap_btree.h" #include "xfs_trace.h" +/* +* Decide if the filesystem has the parent pointer feature or any feature +* added after that. +*/ +static inline bool +xfs_has_parent_or_newer_feature ( +struct xfs_mount *mp) +{ +if (!xfs_sb_is_v5 (&mp->m_sb)) +return false; + +if (xfs_sb_has_compat_feature (&mp->m_sb, ~0 )) +return true; + +if (xfs_sb_has_ro_compat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_RO_COMPAT_FINOBT | +XFS_SB_FEAT_RO_COMPAT_RMAPBT | +XFS_SB_FEAT_RO_COMPAT_REFLINK | +XFS_SB_FEAT_RO_COMPAT_INOBTCNT ))) +return true; + +if (xfs_sb_has_incompat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_INCOMPAT_FTYPE | +XFS_SB_FEAT_INCOMPAT_SPINODES | +XFS_SB_FEAT_INCOMPAT_META_UUID | +XFS_SB_FEAT_INCOMPAT_BIGTIME | +XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR | +XFS_SB_FEAT_INCOMPAT_NREXT64 ))) +return true; + +return false; +} +/* * Calculate the maximum length in bytes that would be required for a local * attribute value as large attributes out of line are not logged.
@@ -31,6 +64,16 @@ xfs_log_calc_max_attrsetm_res(MAXNAMELEN -1 ; nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); nblks +=XFS_B_TO_FSB (mp, size); + +/* +* Starting with the parent pointer feature, every new fs feature +* corrects a unit conversion error in the xattr transaction +* reservation code that resulted in oversized minimum log size +* computations. +*/ +if (xfs_has_parent_or_newer_feature (mp)) + size =XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); return M_RES(mp)->tr_attrsetm.tr_logres +
PATCH v11 15/32 xfs: drop compatibility minimum log size computations for reflink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Having established that we can reduce the minimum log size computation for filesystems with parent pointers or any newer feature, we should also drop the compat minlogsize code that we added when we reduced the transaction reservation size for rmap and reflink.
--- libxfs/xfs_log_rlimit.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
libxfs/xfs_log_rlimit.c +10 -0
@@ -91,6 +91,16 @@ xfs_log_calc_trans_resv_for_minlogblocks( {unsigned int rmap_maxlevels = mp->m_rmap_maxlevels; +/* +* Starting with the parent pointer feature, every new fs feature +* drops the oversized minimum log size computation introduced by the +* original reflink code. +*/ +if (xfs_has_parent_or_newer_feature (mp)) { +xfs_trans_resv_calc (mp, resv); +return ; + } +/* * In the early days of rmap+reflink, we always set the rmap maxlevels * to 9 even if the AG was small enough that it would never grow to
PATCH v11 16/32 xfs: don't remove the attr fork when parent pointers are enabled
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
When an inode is removed, it may also cause the attribute fork to be removed if it is the last attribute. This transaction gets flushed to the log, but if the system goes down before we could inactivate the symlink, the log recovery tries to inactivate this inode (since it is on the unlinked list) but the verifier trips over the remote value and leaks it.
Hence we ended up with a file in this odd state on a "clean" mount. The "obvious" fix is to prohibit erasure of the attr fork to avoid tripping over the verifiers when pptrs are enabled.
--- libxfs/xfs_attr_leaf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
libxfs/xfs_attr_leaf.c +4 -2
@@ -875,7 +875,8 @@ xfs_attr_sf_removename( totsize -= size;if (totsize ==sizeof (xfs_attr_sf_hdr_t) &&xfs_has_attr2 (mp) && (dp->i_df.if_format !=XFS_DINODE_FMT_BTREE ) && - !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE ))) { + !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE )) && + !xfs_has_parent (mp)) {xfs_attr_fork_remove (dp, args->trans); }else {xfs_idata_realloc (dp, -size,XFS_ATTR_FORK );
@@ -884,7 +885,8 @@ xfs_attr_sf_removename(ASSERT (totsize >sizeof (xfs_attr_sf_hdr_t) || (args->op_flags &XFS_DA_OP_ADDNAME ) || !xfs_has_attr2 (mp) || - dp->i_df.if_format ==XFS_DINODE_FMT_BTREE ); + dp->i_df.if_format ==XFS_DINODE_FMT_BTREE || +xfs_has_parent (mp));xfs_trans_log_inode (args->trans, dp,XFS_ILOG_CORE |XFS_ILOG_ADATA ); }
PATCH v11 17/32 xfs: Add the parent pointer support to the superblock version 5.
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the parent pointer superblock flag so that we can actually mount filesystems with this feature enabled.
--- libfrog/fsgeom.c | 4 ++++ libxfs/xfs_format.h | 4 +++- libxfs/xfs_fs.h | 1 + libxfs/xfs_sb.c | 4 ++++ 4 files changed, 12 insertions(+), 1 deletion(-)
libfrog/fsgeom.c +4 -0
@@ -31,6 +31,7 @@ xfs_report_geom(int bigtime_enabled;int inobtcount;int nrext64; +int parent; isint = geo->logstart >0 ; lazycount = geo->flags &XFS_FSOP_GEOM_FLAGS_LAZYSB ?1 :0 ;
@@ -49,12 +50,14 @@ xfs_report_geom( bigtime_enabled = geo->flags &XFS_FSOP_GEOM_FLAGS_BIGTIME ?1 :0 ; inobtcount = geo->flags &XFS_FSOP_GEOM_FLAGS_INOBTCNT ?1 :0 ; nrext64 = geo->flags &XFS_FSOP_GEOM_FLAGS_NREXT64 ?1 :0 ; + parent = geo->flags &XFS_FSOP_GEOM_FLAGS_PARENT ?1 :0 ; printf(_( "meta-data=%-22s isize=%-6d agcount=%u, agsize=%u blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u\n" " =%-22s reflink=%-4u bigtime=%u inobtcount=%u nrext64=%u\n" +" =%-22s parent=%d\n" "data =%-22s bsize=%-6u blocks=%llu, imaxpct=%u\n" " =%-22s sunit=%-6u swidth=%u blks\n" "naming =version %-14u bsize=%-6u ascii-ci=%d, ftype=%d\n"
@@ -65,6 +68,7 @@ xfs_report_geom("" , geo->sectsize, attrversion, projid32bit,"" , crcs_enabled, finobt_enabled, spinodes, rmapbt_enabled,"" , reflink_enabled, bigtime_enabled, inobtcount, nrext64, +"" , parent,"" , geo->blocksize, (unsigned long long )geo->datablocks, geo->imaxpct, "", geo->sunit, geo->swidth,
libxfs/xfs_format.h +3 -1
@@ -373,13 +373,15 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_BIGTIME (1 << 3)/* large timestamps */ #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)/* needs xfs_repair */ #define XFS_SB_FEAT_INCOMPAT_NREXT64 (1 << 5)/* large extent counters */ +#define XFS_SB_FEAT_INCOMPAT_PARENT (1 << 6)/* parent pointers */ #define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ XFS_SB_FEAT_INCOMPAT_META_UUID| \ XFS_SB_FEAT_INCOMPAT_BIGTIME| \ XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \ - XFS_SB_FEAT_INCOMPAT_NREXT64) + XFS_SB_FEAT_INCOMPAT_NREXT64| \ + XFS_SB_FEAT_INCOMPAT_PARENT)#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
libxfs/xfs_fs.h +1 -0
@@ -237,6 +237,7 @@ typedef struct xfs_fsop_resblks {#define XFS_FSOP_GEOM_FLAGS_BIGTIME (1 << 21)/* 64-bit nsec timestamps */ #define XFS_FSOP_GEOM_FLAGS_INOBTCNT (1 << 22)/* inobt btree counter */ #define XFS_FSOP_GEOM_FLAGS_NREXT64 (1 << 23)/* large extent counters */ +#define XFS_FSOP_GEOM_FLAGS_PARENT (1 << 24)/* parent pointers */ /* * Minimum and maximum sizes need for growth checks.
libxfs/xfs_sb.c +4 -0
@@ -171,6 +171,8 @@ xfs_sb_version_to_features( features |=XFS_FEAT_NEEDSREPAIR ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_NREXT64 ) features |=XFS_FEAT_NREXT64 ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_PARENT ) + features |=XFS_FEAT_PARENT ;return features; }
@@ -1188,6 +1190,8 @@ xfs_fs_geometry( geo->flags |=XFS_FSOP_GEOM_FLAGS_BIGTIME ;if (xfs_has_inobtcounts (mp)) geo->flags |=XFS_FSOP_GEOM_FLAGS_INOBTCNT ; +if (xfs_has_parent (mp)) + geo->flags |=XFS_FSOP_GEOM_FLAGS_PARENT ;if (xfs_has_sector (mp)) { geo->flags |=XFS_FSOP_GEOM_FLAGS_SECTOR ; geo->logsectsize = sbp->sb_logsectsize;
PATCH v11 18/32 libfrog: add parent pointer support code
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add some support code to libfrog so that client programs can walk file descriptors and handles upwards through the directory tree; and obtain a reasonable file path from a file descriptor/handle. This code will be used in xfsprogs utilities.
--- include/handle.h | 1 libfrog/Makefile | 2 libfrog/getparents.c | 338 ++++++++++++++++++++++++++++++++++++++++++++++++++ libfrog/getparents.h | 36 +++++ libfrog/paths.c | 167 +++++++++++++++++++++++++ libfrog/paths.h | 25 ++++ libhandle/handle.c | 7 + 7 files changed, 573 insertions(+), 3 deletions(-) create mode 100644 libfrog/getparents.c create mode 100644 libfrog/getparents.h
include/handle.h +1 -0
@@ -17,6 +17,7 @@ struct parent;extern int path_to_handle (char *__path,void **__hanp,size_t *__hlen);extern int path_to_fshandle (char *__path,void **__fshanp,size_t *__fshlen);extern int fd_to_handle (int fd,void **hanp,size_t *hlen); +extern int handle_to_fsfd (void *,char **);extern int handle_to_fshandle (void *__hanp,size_t __hlen,void **__fshanp,size_t *__fshlen);extern void free_handle (void *__hanp,size_t __hlen);
libfrog/Makefile +2 -0
@@ -19,6 +19,7 @@ bulkstat.c \ convert.c \ crc32.c \ fsgeom.c \ +getparents.c \ list_sort.c \ linux.c \ logging.c \
@@ -40,6 +41,7 @@ crc32cselftest.h \ crc32defs.h \ crc32table.h \ fsgeom.h \ +getparents.h \ logging.h \ paths.h \ projects.h \
libfrog/getparents.c new file +338 -0
338 lines not on this page · raw message
libfrog/getparents.h new file +36 -0
@@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __LIBFROG_GETPARENTS_H_ +#define __LIBFROG_GETPARENTS_H_ + +struct path_list ; + +struct parent_rec { +uint64_t p_ino; +uint32_t p_gen; +uint32_t p_flags; +unsigned char p_name[MAXNAMELEN ]; +}; + +/* This is the root directory. */ +#define PARENT_IS_ROOT (1U << 0) + +typedef int (*walk_parent_fn )(const struct parent_rec *rec,void *arg); +typedef int (*walk_path_fn )(const char *mntpt,const struct path_list *path, +void *arg); + +int fd_walk_parents (int fd,walk_parent_fn fn,void *arg); +int handle_walk_parents (void *hanp,size_t hanlen,walk_parent_fn fn, +void *arg); + +int fd_walk_parent_paths (int fd,walk_path_fn fn,void *arg); +int handle_walk_parent_paths (void *hanp,size_t hanlen,walk_path_fn fn, +void *arg); + +int fd_to_path (int fd,char *path,size_t pathlen); +int handle_to_path (void *hanp,size_t hlen,char *path,size_t pathlen); + +#endif /* __LIBFROG_GETPARENTS_H_ */
libfrog/paths.c +167 -0
176 lines not on this page · raw message
libfrog/paths.h +25 -0
@@ -58,4 +58,29 @@ typedef struct fs_cursor {extern void fs_cursor_initialise (char *__dir,uint __flags,fs_cursor_t *__cp);extern fs_path_t *fs_cursor_next_entry (fs_cursor_t *__cp); +/* Path information. */ + +struct path_list ; +struct path_component ; + +struct path_component *path_component_init (const char *name,uint64_t ino); +void path_component_free (struct path_component *pc); +int path_component_change (struct path_component *pc,const void *name, +size_t namelen,uint64_t ino); + +struct path_list *path_list_init (void ); +void path_list_free (struct path_list *path); +void path_list_add_parent_component (struct path_list *path, +struct path_component *pc); +void path_list_add_component (struct path_list *path,struct path_component *pc); +void path_list_del_component (struct path_list *path,struct path_component *pc); + +ssize_t path_list_to_string (const struct path_list *path,char *buf, +size_t buflen); + +typedef int (*path_walk_fn_t )(const char *name,uint64_t ino,void *arg); + +int path_walk_components (const struct path_list *path,path_walk_fn_t fn, +void *arg); +#endif /* __LIBFROG_PATH_H__ */
libhandle/handle.c +4 -3
@@ -29,7 +29,6 @@ typedef union { } comarg_t;static int obj_to_handle (char *,int ,unsigned int ,comarg_t ,void **,size_t *); -static int handle_to_fsfd (void *,char **);static char *path_to_fspath (char *path);
@@ -203,8 +202,10 @@ handle_to_fshandle(return 0 ; } -static int -handle_to_fsfd (void *hanp,char **path) +int +handle_to_fsfd ( +void *hanp, +char **path) {struct fdhash *fdhp;
PATCH v11 19/32 xfs_io: adapt parent command to new parent pointer ioctls
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
For ages, xfs_io has had a totally useless 'parent' command that enabled callers to walk the parents or print the directory tree path of an open file. This code used the ioctl interface presented by SGI's version of parent pointers that was never merged. Rework the code in here to use the new ioctl interfaces that we've settled upon. Get rid of the old parent pointer checking code since xfs_repair/xfs_scrub will take care of that.
(This originally was in the "xfsprogs: implement the upper half of parent pointers" megapatch.)
--- io/parent.c | 470 ++++++++++++++--------------------------------------- man/man8/xfs_io.8 | 21 +- 2 files changed, 133 insertions(+), 358 deletions(-)
io/parent.c +122 -348
@@ -7,365 +7,112 @@#include "command.h" #include "input.h" #include "libfrog/paths.h" -#include "parent.h" +#include "libfrog/getparents.h" #include "handle.h" -#include "jdm.h" #include "init.h" #include "io.h" -#define PARENTBUF_SZ 16384 -#define BSTATBUF_SZ 16384 -static cmdinfo_t parent_cmd; -static int verbose_flag; -static int err_status; -static __u64 inodes_checked;static char *mntpt; -/* -* check out a parent entry to see if the values seem valid -*/ -static void -check_parent_entry (struct xfs_bstat *bstatp,parent_t *parent) +static int +pptr_print ( +const struct parent_rec *rec, +void *arg) { -int sts; -char fullpath[PATH_MAX ]; -struct stat statbuf; -char *str; - -sprintf (fullpath,_ ("%s%s" ), mntpt, parent->p_name); - - sts =lstat (fullpath, &statbuf); -if (sts !=0 ) { -fprintf (stderr, -_ ("inode-path for inode: %llu is incorrect - path \"%s\" non-existent\n" ), - (unsigned long long ) bstatp->bs_ino, fullpath); -if (verbose_flag) { -fprintf (stderr, -_ ("path \"%s\" does not stat for inode: %llu; err = %s\n" ), - fullpath, - (unsigned long long ) bstatp->bs_ino, -strerror (errno)); - } - err_status++; -return ; - }else { -if (verbose_flag >1 ) { -printf (_ ("path \"%s\" found\n" ), fullpath); - } - } - -if (statbuf.st_ino != bstatp->bs_ino) { -fprintf (stderr, -_ ("inode-path for inode: %llu is incorrect - wrong inode#\n" ), - (unsigned long long ) bstatp->bs_ino); -if (verbose_flag) { -fprintf (stderr, -_ ("ino mismatch for path \"%s\" %llu vs %llu\n" ), - fullpath, - (unsigned long long )statbuf.st_ino, - (unsigned long long )bstatp->bs_ino); - } - err_status++; -return ; - }else if (verbose_flag >1 ) { -printf (_ ("inode number match: %llu\n" ), - (unsigned long long )statbuf.st_ino); - } +const char *name = (char *)rec->p_name; +unsigned int namelen; -/* get parent path */ - str =strrchr (fullpath,'/' ); - *str ='\0' ; - sts =stat (fullpath, &statbuf); -if (sts !=0 ) { -fprintf (stderr, -_ ("parent path \"%s\" does not stat: %s\n" ), - fullpath, -strerror (errno)); - err_status++; -return ; - }else { -if (parent->p_ino != statbuf.st_ino) { -fprintf (stderr, -_ ("inode-path for inode: %llu is incorrect - wrong parent inode#\n" ), - (unsigned long long ) bstatp->bs_ino); -if (verbose_flag) { -fprintf (stderr, -_ ("ino mismatch for path \"%s\" %llu vs %llu\n" ), - fullpath, - (unsigned long long )parent->p_ino, - (unsigned long long )statbuf.st_ino); - } - err_status++; -return ; - }else { -if (verbose_flag >1 ) { -printf (_ ("parent ino match for %llu\n" ), - (unsigned long long ) parent->p_ino); - } - } +if (rec->p_flags &PARENT_IS_ROOT ) { +printf (_ ("Root directory.\n" )); +return 0 ; } -} - -static void -check_parents (parent_t *parentbuf,size_t *parentbuf_size, -jdm_fshandle_t *fshandlep,struct xfs_bstat *statp) -{ -int error, i; -__u32 count; -parent_t *entryp; - -do { - error =jdm_parentpaths (fshandlep, statp, parentbuf, *parentbuf_size, &count); - -if (error ==ERANGE ) { - *parentbuf_size *=2 ; - parentbuf = (parent_t *)realloc (parentbuf, *parentbuf_size); - }else if (error) { -fprintf (stderr,_ ("parentpaths failed for ino %llu: %s\n" ), - (unsigned long long ) statp->bs_ino, -strerror (errno)); - err_status++; -break ; - } - }while (error ==ERANGE ); - -if (count ==0 ) { -/* no links for inode - something wrong here */ -fprintf (stderr,_ ("inode-path for inode: %llu is missing\n" ), - (unsigned long long ) statp->bs_ino); - err_status++; - } + namelen =strlen (name); +printf (_ ("p_ino = %llu\n" ), (unsigned long long )rec->p_ino); +printf (_ ("p_gen = %u\n" ), (unsigned int )rec->p_gen); +printf (_ ("p_namelen = %u\n" ), namelen); +printf (_ ("p_name = \"%s\"\n\n" ), rec->p_name); - entryp = parentbuf; -for (i =0 ; i < count; i++) { -check_parent_entry (statp, entryp); - entryp = (parent_t *) (((char *)entryp) + entryp->p_reclen); - } +return 0 ; }static int -do_bulkstat (parent_t *parentbuf,size_t *parentbuf_size, -struct xfs_bstat *bstatbuf,int fsfd,jdm_fshandle_t *fshandlep) +print_parents ( +struct xfs_handle *handle) { -__s32 buflenout; -__u64 lastino =0 ; -struct xfs_bstat *p; -struct xfs_bstat *endp; -struct xfs_fsop_bulkreq bulkreq; -struct stat mntstat; - -if (stat (mntpt, &mntstat)) { -fprintf (stderr,_ ("can't stat mount point \"%s\": %s\n" ), - mntpt,strerror (errno)); -return 1 ; - } - - bulkreq.lastip = &lastino; - bulkreq.icount =BSTATBUF_SZ ; - bulkreq.ubuffer = (void *)bstatbuf; - bulkreq.ocount = &buflenout; - -while (xfsctl (mntpt, fsfd,XFS_IOC_FSBULKSTAT , &bulkreq) ==0 ) { -if (*(bulkreq.ocount) ==0 ) { -return 0 ; - } -for (p = bstatbuf, endp = bstatbuf + *bulkreq.ocount; p < endp; p++) { - -/* inode being modified, get synced data with iget */ -if ( (!p->bs_nlink || !p->bs_mode) && p->bs_ino !=0 ) { - -if (xfsctl (mntpt, fsfd,XFS_IOC_FSBULKSTAT_SINGLE , &bulkreq) <0 ) { -fprintf (stderr, -_ ("failed to get bulkstat information for inode %llu\n" ), - (unsigned long long ) p->bs_ino); -continue ; - } -if (!p->bs_nlink || !p->bs_mode || !p->bs_ino) { -fprintf (stderr, -_ ("failed to get valid bulkstat information for inode %llu\n" ), - (unsigned long long ) p->bs_ino); -continue ; - } - } - -/* skip root */ -if (p->bs_ino == mntstat.st_ino) { -continue ; - } - -if (verbose_flag >1 ) { -printf (_ ("checking inode %llu\n" ), - (unsigned long long ) p->bs_ino); - } - -/* print dotted progress */ -if ((inodes_checked %100 ) ==0 && verbose_flag ==1 ) { -printf ("." );fflush (stdout); - } - inodes_checked++; - -check_parents (parentbuf, parentbuf_size, fshandlep, p); - } +int ret; - }/*while*/ +if (handle) + ret =handle_walk_parents (handle,sizeof (*handle), pptr_print, +NULL ); +else + ret =fd_walk_parents (file->fd, pptr_print,NULL ); +if (ret) +fprintf (stderr,"%s: %s\n" , file->name,strerror (ret)); -fprintf (stderr,_ ("syssgi bulkstat failed: %s\n" ),strerror (errno)); -return 1 ; +return 0 ; }static int -parent_check (void ) +path_print ( +const char *mntpt, +const struct path_list *path, +void *arg) { -int fsfd; -jdm_fshandle_t *fshandlep; -parent_t *parentbuf; -size_t parentbuf_size =PARENTBUF_SZ ; -struct xfs_bstat *bstatbuf; - - err_status =0 ; - inodes_checked =0 ; - -sync (); - - fsfd = file->fd; - - fshandlep =jdm_getfshandle (mntpt); -if (fshandlep ==NULL ) { -fprintf (stderr,_ ("unable to open \"%s\" for jdm: %s\n" ), - mntpt, -strerror (errno)); -return 1 ; - } +char buf[PATH_MAX ]; +size_t len =PATH_MAX ; +int mntpt_len =strlen (mntpt); +int ret; -/* allocate buffers */ - bstatbuf = (struct xfs_bstat *)calloc (BSTATBUF_SZ ,sizeof (struct xfs_bstat )); - parentbuf = (parent_t *)malloc (parentbuf_size); -if (!bstatbuf || !parentbuf) { -fprintf (stderr,_ ("unable to allocate buffers: %s\n" ), -strerror (errno)); - err_status =1 ; - goto out; - } - -if (do_bulkstat (parentbuf, &parentbuf_size, bstatbuf, fsfd, fshandlep) !=0 ) - err_status++; +/* Trim trailing slashes from the mountpoint */ +while (mntpt_len >0 && mntpt[mntpt_len -1 ] =='/' ) + mntpt_len--; -if (err_status >0 ) -fprintf (stderr,_ ("num errors: %d\n" ), err_status); -else -printf (_ ("succeeded checking %llu inodes\n" ), - (unsigned long long ) inodes_checked); + ret =snprintf (buf, len,"%.*s" , mntpt_len, mntpt); +if (ret != mntpt_len) +return ENAMETOOLONG ; -out: -free (bstatbuf); -free (parentbuf); -free (fshandlep); -return err_status; -} + ret =path_list_to_string (path, buf + ret, len - ret); +if (ret <0 ) +return ENAMETOOLONG ; -static void -print_parent_entry (parent_t *parent,int fullpath) -{ -printf (_ ("p_ino = %llu\n" ), (unsigned long long ) parent->p_ino); -printf (_ ("p_gen = %u\n" ), parent->p_gen); -printf (_ ("p_reclen = %u\n" ), parent->p_reclen); -if (fullpath) -printf (_ ("p_name = \"%s%s\"\n" ), mntpt, parent->p_name); -else -printf (_ ("p_name = \"%s\"\n" ), parent->p_name); +printf ("%s\n" , buf); +return 0 ; }static int -parent_list (int fullpath) +print_paths ( +struct xfs_handle *handle) { -void *handlep =NULL ; -size_t handlen; -int error, i; -int retval =1 ; -__u32 count; -parent_t *entryp; -parent_t *parentbuf =NULL ; -char *path = file->name; -int pb_size =PARENTBUF_SZ ; +int ret; -/* XXXX for linux libhandle version - to set libhandle fsfd cache */ - { -void *fshandle; -size_t fshlen; - -if (path_to_fshandle (mntpt, &fshandle, &fshlen) !=0 ) { -fprintf (stderr,_ ("%s: failed path_to_fshandle \"%s\": %s\n" ), - progname, path,strerror (errno)); - goto error; - } -free_handle (fshandle, fshlen); - } - -if (path_to_handle (path, &handlep, &handlen) !=0 ) { -fprintf (stderr,_ ("%s: path_to_handle failed for \"%s\"\n" ), progname, path); - goto error; - } - -do { - parentbuf = (parent_t *)realloc (parentbuf, pb_size); -if (!parentbuf) { -fprintf (stderr,_ ("%s: unable to allocate parent buffer: %s\n" ), - progname,strerror (errno)); - goto error; - } - -if (fullpath) { - error =parentpaths_by_handle (handlep, - handlen, - parentbuf, - pb_size, - &count); - }else { - error =parents_by_handle (handlep, - handlen, - parentbuf, - pb_size, - &count); - } -if (error ==ERANGE ) { - pb_size *=2 ; - }else if (error) { -fprintf (stderr,_ ("%s: %s call failed for \"%s\": %s\n" ), - progname, fullpath ?"parentpaths" :"parents" , - path,strerror (errno)); - goto error; - } - }while (error ==ERANGE ); - -if (count ==0 ) { -/* no links for inode - something wrong here */ -fprintf (stderr,_ ("%s: inode-path is missing\n" ), progname); - goto error; - } - - entryp = parentbuf; -for (i =0 ; i < count; i++) { -print_parent_entry (entryp, fullpath); - entryp = (parent_t *) (((char *)entryp) + entryp->p_reclen); - } - - retval =0 ; -error: -free (handlep); -free (parentbuf); -return retval; +if (handle) + ret =handle_walk_parent_paths (handle,sizeof (*handle), + path_print,NULL ); +else + ret =fd_walk_parent_paths (file->fd, path_print,NULL ); +if (ret) +fprintf (stderr,"%s: %s\n" , file->name,strerror (ret)); +return 0 ; }static int -parent_f (int argc,char **argv) +parent_f ( +int argc, +char **argv) { -int c; -int listpath_flag =0 ; -int check_flag =0 ; -fs_path_t *fs; -static int tab_init; +struct xfs_handle handle; +void *hanp =NULL ; +size_t hlen; +struct fs_path *fs; +char *p; +uint64_t ino =0 ; +uint32_t gen =0 ; +int c; +int listpath_flag =0 ; +int ret; +static int tab_init;if (!tab_init) { tab_init =1 ;
@@ -380,46 +127,74 @@ parent_f(int argc, char **argv) } mntpt = fs->fs_dir; - verbose_flag =0 ; - -while ((c =getopt (argc, argv,"cpv" )) !=EOF ) { +while ((c =getopt (argc, argv,"p" )) !=EOF ) {switch (c) { -case 'c' : - check_flag =1 ; -break ;case 'p' : listpath_flag =1 ;break ; -case 'v' : - verbose_flag++; -break ;default :return command_usage (&parent_cmd); } } -if (!check_flag && !listpath_flag)/* default case */ - exitcode =parent_list (listpath_flag); -else { -if (listpath_flag) - exitcode =parent_list (listpath_flag); -if (check_flag) - exitcode =parent_check (); +/* +* Always initialize the fshandle table because we need it for +* the ppaths functions to work. +*/ + ret =path_to_fshandle ((char *)mntpt, &hanp, &hlen); +if (ret) { +perror (mntpt); +return 0 ; } +if (optind +2 == argc) { + ino =strtoull (argv[optind], &p,0 ); +if (*p !='\0' || ino ==0 ) { +fprintf (stderr, +_ ("Bad inode number '%s'.\n" ), + argv[optind]); +return 0 ; + } + gen =strtoul (argv[optind +1 ], &p,0 ); +if (*p !='\0' ) { +fprintf (stderr, +_ ("Bad generation number '%s'.\n" ), + argv[optind +1 ]); +return 0 ; + } + +memcpy (&handle, hanp,sizeof (handle)); + handle.ha_fid.fid_len =sizeof (xfs_fid_t) - +sizeof (handle.ha_fid.fid_len); + handle.ha_fid.fid_pad =0 ; + handle.ha_fid.fid_ino = ino; + handle.ha_fid.fid_gen = gen; + + }else if (optind != argc) { +return command_usage (&parent_cmd); + } + +if (listpath_flag) + exitcode =print_paths (ino ? &handle :NULL ); +else + exitcode =print_parents (ino ? &handle :NULL ); + +if (hanp) +free_handle (hanp, hlen); +return 0 ; }static void parent_help (void ) { -printf (_ ( +printf (_ ("\n" " list the current file's parents and their filenames\n" "\n" -" -c -- check the current file's file system for parent consistency\n" -" -p -- list the current file's parents and their full paths\n" -" -v -- verbose mode\n" +" -p -- list the current file's paths up to the root\n" +"\n" +"If ino and gen are supplied, use them instead.\n" "\n" )); }
@@ -430,11 +205,10 @@ parent_init(void) parent_cmd.cfunc = parent_f; parent_cmd.argmin =0 ; parent_cmd.argmax =-1 ; - parent_cmd.args =_ ("[-cpv]" ); + parent_cmd.args =_ ("[-p] [ino gen]" ); parent_cmd.flags =CMD_NOMAP_OK ; - parent_cmd.oneline =_ ("print or check parent inodes" ); + parent_cmd.oneline =_ ("print parent inodes" ); parent_cmd.help = parent_help; -if (expert) -add_command (&parent_cmd); +add_command (&parent_cmd); }
man/man8/xfs_io.8 +11 -10
@@ -961,25 +961,26 @@ and options behave as described above, in .B chproj. .TP -.BR parent " [ " \-cpv " ]" +.BR parent " [ " \-p " ] [" " ino gen " "]" By default this command prints out the parent inode numbers, inode generation numbers and basenames of all the hardlinks which point to the inode of the current file. + +If the optional +.B ino +and +.B gen +parameters are provided, they will be used to create a file handle on the same +filesystem as the open file. +The parents of the file represented by the handle will be reported instead of +the open file. + .RS 1.0i .PD 0 .TP 0.4i .B \-p the output is similar to the default output except pathnames up to the mount-point are printed out instead of the component name. -.TP -.B \-c -the file's filesystem will check all the parent attributes for consistency. -.TP -.B \-v -verbose output will be printed. -.RE -.IP -.B [NOTE: Not currently operational on Linux.] .RE .PD .TP
PATCH v11 20/32 xfs_io: Add i, n and f flags to parent command
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
This patch adds the flags i, n, and f to the parent command. These flags add filtering options that are used by the new parent pointer tests in xfstests, and help to improve the test run time. The flags are:
-i: Only show parent pointer records containing the given inode -n: Only show parent pointer records containing the given filename -f: Print records in short format: ino/gen/namelen/name
--- io/parent.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++------ man/man8/xfs_io.8 | 11 ++++++- 2 files changed, 89 insertions(+), 11 deletions(-)
io/parent.c +79 -10
@@ -15,11 +15,18 @@static cmdinfo_t parent_cmd;static char *mntpt; +struct pptr_args { +uint64_t filter_ino; +char *filter_name; +bool shortformat; +}; +static int pptr_print (const struct parent_rec *rec,void *arg) { +struct pptr_args *args = arg;const char *name = (char *)rec->p_name;unsigned int namelen;
@@ -28,7 +35,22 @@ pptr_print(return 0 ; } +if (args->filter_ino && rec->p_ino != args->filter_ino) +return 0 ; +if (args->filter_name &&strcmp (args->filter_name, name)) +return 0 ; + namelen =strlen (name); + +if (args->shortformat) { +printf ("%llu/%u/%u/%s\n" , + (unsigned long long )rec->p_ino, + (unsigned int )rec->p_gen, + namelen, + rec->p_name); +return 0 ; + } +printf (_ ("p_ino = %llu\n" ), (unsigned long long )rec->p_ino);printf (_ ("p_gen = %u\n" ), (unsigned int )rec->p_gen);printf (_ ("p_namelen = %u\n" ), namelen);
@@ -39,32 +61,55 @@ pptr_print(static int print_parents ( -struct xfs_handle *handle) +struct xfs_handle *handle, +struct pptr_args *args) {int ret;if (handle) ret =handle_walk_parents (handle,sizeof (*handle), pptr_print, -NULL ); + args);else - ret =fd_walk_parents (file->fd, pptr_print,NULL ); + ret =fd_walk_parents (file->fd, pptr_print, args);if (ret)fprintf (stderr,"%s: %s\n" , file->name,strerror (ret));return 0 ; } +static int +filter_path_components ( +const char *name, +uint64_t ino, +void *arg) +{ +struct pptr_args *args = arg; + +if (args->filter_ino && ino == args->filter_ino) +return ECANCELED ; +if (args->filter_name && !strcmp (args->filter_name, name)) +return ECANCELED ; +return 0 ; +} +static int path_print (const char *mntpt,const struct path_list *path,void *arg) { +struct pptr_args *args = arg;char buf[PATH_MAX ];size_t len =PATH_MAX ;int mntpt_len =strlen (mntpt);int ret; +if (args->filter_ino || args->filter_name) { + ret =path_walk_components (path, filter_path_components, args); +if (ret !=ECANCELED ) +return 0 ; + } +/* Trim trailing slashes from the mountpoint */ while (mntpt_len >0 && mntpt[mntpt_len -1 ] =='/' ) mntpt_len--;
@@ -83,15 +128,16 @@ path_print(static int print_paths ( -struct xfs_handle *handle) +struct xfs_handle *handle, +struct pptr_args *args) {int ret;if (handle) ret =handle_walk_parent_paths (handle,sizeof (*handle), - path_print,NULL ); + path_print, args);else - ret =fd_walk_parent_paths (file->fd, path_print,NULL ); + ret =fd_walk_parent_paths (file->fd, path_print, args);if (ret)fprintf (stderr,"%s: %s\n" , file->name,strerror (ret));return 0 ;
@@ -103,6 +149,7 @@ parent_f( char **argv) {struct xfs_handle handle; +struct pptr_args args = {0 };void *hanp =NULL ;size_t hlen;struct fs_path *fs;
@@ -127,11 +174,27 @@ parent_f( } mntpt = fs->fs_dir; -while ((c =getopt (argc, argv,"p" )) !=EOF ) { +while ((c =getopt (argc, argv,"pfi:n:" )) !=EOF ) {switch (c) {case 'p' : listpath_flag =1 ;break ; +case 'i' : + args.filter_ino =strtoull (optarg, &p,0 ); +if (*p !='\0' || args.filter_ino ==0 ) { +fprintf (stderr, +_ ("Bad inode number '%s'.\n" ), + optarg); +return 0 ; + } + +break ; +case 'n' : + args.filter_name = optarg; +break ; +case 'f' : + args.shortformat = true; +break ;default :return command_usage (&parent_cmd); }
@@ -175,9 +238,9 @@ parent_f( }if (listpath_flag) - exitcode =print_paths (ino ? &handle :NULL ); + exitcode =print_paths (ino ? &handle :NULL , &args);else - exitcode =print_parents (ino ? &handle :NULL ); + exitcode =print_parents (ino ? &handle :NULL , &args);if (hanp)free_handle (hanp, hlen);
@@ -195,6 +258,12 @@ printf(_(" -p -- list the current file's paths up to the root\n" "\n" "If ino and gen are supplied, use them instead.\n" +"\n" +" -i -- Only show parent pointer records containing the given inode\n" +"\n" +" -n -- Only show parent pointer records containing the given filename\n" +"\n" +" -f -- Print records in short format: ino/gen/namelen/filename\n" "\n")); }
@@ -205,7 +274,7 @@ parent_init(void) parent_cmd.cfunc = parent_f; parent_cmd.argmin =0 ; parent_cmd.argmax =-1 ; - parent_cmd.args =_ ("[-p] [ino gen]" ); + parent_cmd.args =_ ("[-p] [ino gen] [-i ino] [-n name] [-f]" ); parent_cmd.flags =CMD_NOMAP_OK ; parent_cmd.oneline =_ ("print parent inodes" ); parent_cmd.help = parent_help;
man/man8/xfs_io.8 +10 -1
@@ -961,7 +961,7 @@ and options behave as described above, in .B chproj. .TP -.BR parent " [ " \-p " ] [" " ino gen " "]" +.BR parent " [ " \-fp " ] [-i " ino "] [-n " name "] [" " ino gen " "]" By default this command prints out the parent inode numbers, inode generation numbers and basenames of all the hardlinks which point to the inode of the current file.
@@ -978,6 +978,15 @@ the open file. .RS 1.0i .PD 0 .TP 0.4i +.B \-f +Print records in short format: ino/gen/namelen/name +.TP 0.4i +.B \-i +Only show parent pointer records containing this inode number. +.TP 0.4i +.B \-n +Only show parent pointer records containing this directory entry name. +.TP 0.4i .B \-p the output is similar to the default output except pathnames up to the mount-point are printed out instead of the component name.
PATCH v11 21/32 xfs_logprint: decode parent pointers in ATTRI items fully
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
This patch modifies the ATTRI print routines to look for the parent pointer flag, and decode logged parent pointers fully when dumping log contents. Between the existing ATTRI: printouts and the new ones introduced here, we can figure out what was stored in each log iovec, as well as the higher level parent pointer that was logged.
--- libxfs/libxfs_api_defs.h | 4 ++ logprint/log_redo.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 85 insertions(+)
libxfs/libxfs_api_defs.h +4 -0
@@ -139,6 +139,10 @@#define xfs_log_get_max_trans_res libxfs_log_get_max_trans_res#define xfs_log_sb libxfs_log_sb#define xfs_mode_to_ftype libxfs_mode_to_ftype +#define xfs_parent_add libxfs_parent_add +#define xfs_parent_finish libxfs_parent_finish +#define xfs_parent_irec_from_disk libxfs_parent_irec_from_disk +#define xfs_parent_start libxfs_parent_start#define xfs_perag_get libxfs_perag_get#define xfs_perag_put libxfs_perag_put#define xfs_prealloc_blocks libxfs_prealloc_blocks
logprint/log_redo.c +81 -0
155 lines not on this page · raw message
PATCH v11 22/32 xfs_scrub: use parent pointers when possible to report file operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If parent pointers are available, use them to supply file paths when doing things to files, instead of merely printing the inode number.
--- scrub/common.c | 41 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-)
scrub/common.c +39 -2
@@ -9,6 +9,7 @@#include <syslog.h> #include "platform_defs.h" #include "libfrog/paths.h" +#include "libfrog/getparents.h" #include "xfs_scrub.h" #include "common.h" #include "progress.h"
@@ -403,19 +404,55 @@ scrub_render_ino_descr( ...) {va_list args; +size_t pathlen =0 ;uint32_t agno;uint32_t agino;int ret; +if (ctx->mnt.fsgeom.flags &XFS_FSOP_GEOM_FLAGS_PARENT ) { +struct xfs_handle handle; + +memcpy (&handle.ha_fsid, ctx->fshandle,sizeof (handle.ha_fsid)); + handle.ha_fid.fid_len =sizeof (xfs_fid_t) - +sizeof (handle.ha_fid.fid_len); + handle.ha_fid.fid_pad =0 ; + handle.ha_fid.fid_ino = ino; + handle.ha_fid.fid_gen = gen; + + ret =handle_to_path (&handle,sizeof (struct xfs_handle ), buf, + buflen); +if (ret) + goto report_inum; + +/* +* Leave at least 16 bytes for the description of what went +* wrong. If we can't do that, we'll use the inode number. +*/ + pathlen =strlen (buf); +if (pathlen >= buflen -16 ) + goto report_inum; + +if (format) { + buf[pathlen] =' ' ; + buf[pathlen +1 ] =0 ; + pathlen++; + } + + goto report_format; + } + +report_inum: agno =cvt_ino_to_agno (&ctx->mnt, ino); agino =cvt_ino_to_agino (&ctx->mnt, ino); ret =snprintf (buf, buflen,_ ("inode %" PRIu64" (%" PRIu32"/%" PRIu32")%s" ), ino, agno, agino, format ?" " :"" );if (ret <0 || ret >= buflen || format ==NULL )return ret; + pathlen = ret; +report_format:va_start (args, format); - ret +=vsnprintf (buf + ret, buflen - ret, format, args); + pathlen +=vsnprintf (buf + pathlen, buflen - pathlen, format, args);va_end (args); -return ret; +return pathlen; }
PATCH v11 23/32 xfs_db: report parent bit on xattrs
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Display the parent bit on xattr keys
--- db/attr.c | 3 +++ db/attrshort.c | 3 +++ 2 files changed, 6 insertions(+)
db/attr.c +3 -0
@@ -82,6 +82,9 @@ const field_t attr_leaf_entry_flds[] = { {"local" ,FLDT_UINT1 ,OI (LEOFF (flags) +bitsz (uint8_t) -XFS_ATTR_LOCAL_BIT -1 ),C1 ,0 , TYP_NONE }, + { "parent", FLDT_UINT1, +OI (LEOFF (flags) +bitsz (uint8_t) -XFS_ATTR_PARENT_BIT -1 ),C1 ,0 , + TYP_NONE }, { "pad2", FLDT_UINT8X, OI(LEOFF(pad2)), C1, FLD_SKIPALL, TYP_NONE }, { NULL } };
db/attrshort.c +3 -0
@@ -44,6 +44,9 @@ const field_t attr_sf_entry_flds[] = { {"secure" ,FLDT_UINT1 ,OI (EOFF (flags) +bitsz (uint8_t) -XFS_ATTR_SECURE_BIT -1 ),C1 ,0 , TYP_NONE }, + { "parent", FLDT_UINT1, +OI (EOFF (flags) +bitsz (uint8_t) -XFS_ATTR_PARENT_BIT -1 ),C1 ,0 , + TYP_NONE }, { "name", FLDT_CHARNS, OI(EOFF(nameval)), attr_sf_entry_name_count, FLD_COUNT, TYP_NONE }, { "value", FLDT_CHARNS, attr_sf_entry_value_offset,
PATCH v11 24/32 xfs_db: report parent pointers embedded in xattrs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Decode the parent pointer inode, generation, namehash, and name fields if the parent pointer passes basic validation checks.
--- db/attr.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ db/attrshort.c | 48 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+)
db/attr.c +62 -0
@@ -19,6 +19,8 @@ static int attr_leaf_entries_count(void *obj, int startoff);static int attr_leaf_hdr_count (void *obj,int startoff);static int attr_leaf_name_local_count (void *obj,int startoff);static int attr_leaf_name_local_name_count (void *obj,int startoff); +static int attr_leaf_name_pptr_count (void *obj,int startoff); +static int attr_leaf_name_pptr_namelen (void *obj,int startoff);static int attr_leaf_name_local_value_count (void *obj,int startoff);static int attr_leaf_name_local_value_offset (void *obj,int startoff,int idx);
@@ -111,6 +113,8 @@ const field_t attr_leaf_map_flds[] = {#define LNOFF (f) bitize(offsetof(xfs_attr_leaf_name_local_t, f))#define LVOFF (f) bitize(offsetof(xfs_attr_leaf_name_remote_t, f)) +#define PPOFF (f) bitize(offsetof(xfs_attr_leaf_name_local_t, nameval) + \ + offsetof(struct xfs_parent_name_rec, f))const field_t attr_leaf_name_flds[] = { {"valuelen" ,FLDT_UINT16D ,OI (LNOFF (valuelen)), attr_leaf_name_local_count,FLD_COUNT ,TYP_NONE },
@@ -118,6 +122,14 @@ const field_t attr_leaf_name_flds[] = { attr_leaf_name_local_count, FLD_COUNT, TYP_NONE }, {"name" ,FLDT_CHARNS ,OI (LNOFF (nameval)), attr_leaf_name_local_name_count, FLD_COUNT, TYP_NONE }, + { "parent_ino", FLDT_INO, OI(PPOFF(p_ino)), + attr_leaf_name_pptr_count, FLD_COUNT, TYP_INODE }, + { "parent_gen", FLDT_UINT32D, OI(PPOFF(p_gen)), + attr_leaf_name_pptr_count, FLD_COUNT, TYP_NONE }, + { "parent_namehash", FLDT_UINT32X, OI(PPOFF(p_namehash)), + attr_leaf_name_pptr_count, FLD_COUNT, TYP_NONE }, + { "parent_name", FLDT_CHARNS, attr_leaf_name_local_value_offset, + attr_leaf_name_pptr_namelen, FLD_COUNT|FLD_OFFSET, TYP_NONE }, { "value", FLDT_CHARNS, attr_leaf_name_local_value_offset, attr_leaf_name_local_value_count, FLD_COUNT|FLD_OFFSET, TYP_NONE }, { "valueblk", FLDT_UINT32X, OI(LVOFF(valueblk)),
@@ -273,6 +285,26 @@ attr_leaf_name_local_count( __attr_leaf_name_local_count); } +static int +__attr_leaf_name_pptr_count ( +struct xfs_attr_leafblock *leaf, +struct xfs_attr_leaf_entry *e, +int i) +{ +if ((e->flags &XFS_ATTR_NSP_ONDISK_MASK ) ==XFS_ATTR_PARENT ) +return 1 ; +return 0 ; +} + +static int +attr_leaf_name_pptr_count ( +void *obj, +int startoff) +{ +return attr_leaf_entry_walk (obj, startoff, + __attr_leaf_name_pptr_count); +} + static int __attr_leaf_name_local_name_count( struct xfs_attr_leafblock *leaf,
@@ -283,6 +315,8 @@ __attr_leaf_name_local_name_count(if (!(e->flags &XFS_ATTR_LOCAL ))return 0 ; +if ((e->flags &XFS_ATTR_NSP_ONDISK_MASK ) ==XFS_ATTR_PARENT ) +return 0 ; l =xfs_attr3_leaf_name_local (leaf, i);return l->namelen;
@@ -297,6 +331,32 @@ attr_leaf_name_local_name_count( __attr_leaf_name_local_name_count); } +static int +__attr_leaf_name_pptr_namelen ( +struct xfs_attr_leafblock *leaf, +struct xfs_attr_leaf_entry *e, +int i) +{ +struct xfs_attr_leaf_name_local *l; + +if (!(e->flags &XFS_ATTR_LOCAL )) +return 0 ; +if ((e->flags &XFS_ATTR_NSP_ONDISK_MASK ) !=XFS_ATTR_PARENT ) +return 0 ; + + l =xfs_attr3_leaf_name_local (leaf, i); +return be16_to_cpu (l->valuelen); +} + +static int +attr_leaf_name_pptr_namelen ( +void *obj, +int startoff) +{ +return attr_leaf_entry_walk (obj, startoff, + __attr_leaf_name_pptr_namelen); +} + static int __attr_leaf_name_local_value_count( struct xfs_attr_leafblock *leaf,
@@ -307,6 +367,8 @@ __attr_leaf_name_local_value_count(if (!(e->flags &XFS_ATTR_LOCAL ))return 0 ; +if ((e->flags &XFS_ATTR_NSP_ONDISK_MASK ) ==XFS_ATTR_PARENT ) +return 0 ; l =xfs_attr3_leaf_name_local (leaf, i);return be16_to_cpu (l->valuelen);
db/attrshort.c +48 -0
89 lines not on this page · raw message
PATCH v11 25/32 xfs_db: obfuscate dirent and parent pointer names consistently
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When someone wants to perform an obfuscated metadump of a filesystem where parent pointers are enabled, we have to use the *exact* same obfuscated name for both the directory entry and the parent pointer.
Create a name remapping table so that when we obfuscate a dirent name or a parent pointer name, we can apply the same obfuscation when we find the corresponding parent pointer or dirent.
--- db/metadump.c | 313 ++++++++++++++++++++++++++++++++++++++++++++-- libxfs/libxfs_api_defs.h | 2 2 files changed, 304 insertions(+), 11 deletions(-)
db/metadump.c +302 -11
@@ -20,6 +20,14 @@#include "field.h" #include "dir2.h" +#undef REMAP_DEBUG + +#ifdef REMAP_DEBUG +# define remap_debug printf +#else +# define remap_debug (...) ((void)0) +#endif +#define DEFAULT_MAX_EXT_SIZE XFS_MAX_BMBT_EXTLEN/* copy all metadata structures to/from a file */
@@ -736,6 +744,111 @@ nametable_add(xfs_dahash_t hash, int namelen, unsigned char *name)return ent; } +/* +* Obfuscated name remapping table for parent pointer-enabled filesystems. +* When this feature is enabled, we have to maintain consistency between the +* names that appears in the dirent and the corresponding parent pointer. +*/ + +struct remap_ent { +struct remap_ent *next; +xfs_ino_t dir_ino; +xfs_dahash_t namehash; +uint8_t namelen; + +uint8_t names[]; +}; + +static inline uint8_t *remap_ent_before (struct remap_ent *ent) +{ +return &ent->names[0 ]; +} + +static inline uint8_t *remap_ent_after (struct remap_ent *ent) +{ +return &ent->names[ent->namelen]; +} + +#define REMAP_TABLE_SIZE 4096 + +static struct remap_ent *remaptable[REMAP_TABLE_SIZE ]; + +static void +remaptable_clear (void ) +{ +int i; +struct remap_ent *ent, *next; + +for (i =0 ; i <REMAP_TABLE_SIZE ; i++) { + ent = remaptable[i]; + +while (ent) { + next = ent->next; +free (ent); + ent = next; + } + } +} + +/* Try to find a remapping table entry. */ +static struct remap_ent * +remaptable_find ( +xfs_ino_t dir_ino, +xfs_dahash_t namehash, +const unsigned char *name, +unsigned int namelen) +{ +struct remap_ent *ent = remaptable[namehash %REMAP_TABLE_SIZE ]; + +remap_debug ("REMAP FIND: 0x%lx hash 0x%x '%.*s'\n" , + dir_ino, namehash, namelen, name); + +while (ent) { +remap_debug ("REMAP ENT: 0x%lx hash 0x%x '%.*s'\n" , + ent->dir_ino, ent->namehash, ent->namelen, +remap_ent_before (ent)); + +if (ent->dir_ino == dir_ino && + ent->namehash == namehash && + ent->namelen == namelen && + !memcmp (remap_ent_before (ent), name, namelen)) +return ent; + ent = ent->next; + } + +return NULL ; +} + +/* Remember the remapping for a particular dirent that we obfuscated. */ +static struct remap_ent * +remaptable_add ( +xfs_ino_t dir_ino, +xfs_dahash_t namehash, +const unsigned char *old_name, +unsigned int namelen, +const unsigned char *new_name) +{ +struct remap_ent *ent; + + ent =malloc (sizeof (struct remap_ent ) + (namelen *2 )); +if (!ent) +return NULL ; + + ent->dir_ino = dir_ino; + ent->namehash = namehash; + ent->namelen = namelen; +memcpy (remap_ent_before (ent), old_name, namelen); +memcpy (remap_ent_after (ent), new_name, namelen); + ent->next = remaptable[namehash %REMAP_TABLE_SIZE ]; + + remaptable[namehash %REMAP_TABLE_SIZE ] = ent; + +remap_debug ("REMAP ADD: 0x%lx hash 0x%x '%.*s' -> '%.*s'\n" , + dir_ino, namehash, namelen, old_name, namelen, + new_name); +return ent; +} +#define is_invalid_char (c) ((c) == '/' || (c) == '\0')#define rol32 (x,y) (((x) << (y)) | ((x) >> (32 - (y))))
@@ -1184,6 +1297,7 @@ generate_obfuscated_name(int namelen, unsigned char *name) { + unsigned char *old_name = NULL;xfs_dahash_t hash;/*
@@ -1205,8 +1319,32 @@ generate_obfuscated_name( name++;/* Obfuscate the name (if possible) */ - hash =libxfs_da_hashname (name, namelen); + +/* +* If we're obfuscating a dirent name on a pptrs filesystem, see if we +* already processed the parent pointer and use the same name. +*/ +if (xfs_has_parent (mp) && ino) { +struct remap_ent *remap; + + remap =remaptable_find (cur_ino, hash, name, namelen); +if (remap) { +remap_debug ("found obfuscated dir 0x%lx '%.*s' -> 0x%lx -> '%.*s' \n" , + cur_ino, namelen, +remap_ent_before (remap), ino, namelen, +remap_ent_after (remap)); +memcpy (name,remap_ent_after (remap), namelen); +return ; + } + +/* +* If we haven't procesed this dirent name before, save the +* old name for a remap table entry. Obfuscate the name. +*/ + old_name =alloca (namelen); +memcpy (old_name, name, namelen); + }obfuscate_name (hash, namelen, name);/*
@@ -1230,6 +1368,23 @@ generate_obfuscated_name("in dir inode %llu\n" , (unsigned long long ) ino, (unsigned long long) cur_ino); + +/* +* We've obfuscated a name in the directory entry. Remember this +* remapping for when we come across the parent pointer later. +*/ +if (!old_name) +return ; + +remap_debug ("obfuscating dir 0x%lx '%.*s' -> 0x%lx -> '%.*s' \n" , + cur_ino, namelen, old_name, ino, namelen, + name); + +if (!remaptable_add (cur_ino, hash, old_name, namelen, name)) +print_warning ("unable to record remapped dirent name for inode %llu " +"in dir inode %llu\n" , + (unsigned long long ) ino, + (unsigned long long ) cur_ino); } static void
@@ -1361,6 +1516,123 @@ process_sf_symlink(memset (&buf[len],0 ,XFS_DFORK_DSIZE (dip, mp) - len); } +static inline bool +want_obfuscate_pptr ( +unsigned int nsp_flags, +const void *name, +unsigned int namelen, +const void *value, +unsigned int valuelen) +{ +if (!obfuscate) +return false; + +/* Ignore if parent pointers aren't enabled. */ +if (!xfs_has_parent (mp)) +return false; + +/* Ignore anything not claiming to be a parent pointer. */ +if (!(nsp_flags &XFS_ATTR_PARENT )) +return false; + +/* Obfuscate this parent pointer if it passes basic checks. */ +if (libxfs_parent_namecheck (mp, name, namelen, nsp_flags) && +libxfs_parent_valuecheck (mp, value, valuelen)) +return true; + +/* Ignore otherwise. */ +return false; +} + +static void +obfuscate_parent_pointer ( +const struct xfs_parent_name_rec *rec, +unsigned char *value, +unsigned int valuelen) +{ +struct xfs_parent_name_irec irec; +struct remap_ent *remap; +char *old_name = irec.p_name; +xfs_dahash_t hash; +xfs_ino_t child_ino = cur_ino; + +libxfs_parent_irec_from_disk (&irec, rec, value, valuelen); + +/* +* We don't obfuscate "lost+found" or any orphan files +* therein. If When the name table is used for extended +* attributes, the inode number provided is 0, in which +* case we don't need to make this check. +*/ + cur_ino = irec.p_ino; +if (in_lost_found (child_ino, valuelen, value)) { + cur_ino = child_ino; +return ; + } + cur_ino = child_ino; + +/* +* If the name starts with a slash, just skip over it. It isn't +* included in the hash and we don't record it in the name table. +*/ +if (*value =='/' ) { + old_name++; + value++; + valuelen--; + } + + hash =libxfs_da_hashname (value, valuelen); + +/* +* If we already processed the dirent, use the same name for the parent +* pointer. +*/ + remap =remaptable_find (irec.p_ino, hash, value, valuelen); +if (remap) { +remap_debug ("found obfuscated pptr 0x%lx '%.*s' -> 0x%lx -> '%.*s' \n" , + irec.p_ino, valuelen,remap_ent_before (remap), + cur_ino, valuelen,remap_ent_after (remap)); +memcpy (value,remap_ent_after (remap), valuelen); +return ; + } + +/* +* Obfuscate the parent pointer name and remember this for later +* in case we encounter the dirent and need to reuse the name there. +*/ +obfuscate_name (hash, valuelen, value); + +remap_debug ("obfuscated pptr 0x%lx '%.*s' -> 0x%lx -> '%.*s'\n" , + irec.p_ino, valuelen, old_name, cur_ino, valuelen, + value); +if (!remaptable_add (irec.p_ino, hash, old_name, valuelen, value)) +print_warning ("unable to record remapped pptr name for inode %llu " +"in dir inode %llu\n" , + (unsigned long long ) cur_ino, + (unsigned long long ) irec.p_ino); +} + +static inline bool +want_obfuscate_attr ( +unsigned int nsp_flags, +const void *name, +unsigned int namelen, +const void *value, +unsigned int valuelen) +{ +if (!obfuscate) +return false; + +/* +* If we didn't already obfuscate the parent pointer, it's probably +* corrupt. Leave it intact for analysis. +*/ +if (nsp_flags &XFS_ATTR_PARENT ) +return false; + +return true; +} +static void process_sf_attr (struct xfs_dinode *dip)
@@ -1390,7 +1662,7 @@ process_sf_attr( asfep = &asfp->list[0 ];for (i =0 ; (i < asfp->hdr.count) && ((char *)asfep - (char *)asfp < ino_attr_size); i++) { - +void *name, *value;int namelen = asfep->namelen;if (namelen ==0 ) {
@@ -1406,11 +1678,16 @@ process_sf_attr(break ; } -if (obfuscate) { -generate_obfuscated_name (0 , asfep->namelen, - &asfep->nameval[0 ]); -memset (&asfep->nameval[asfep->namelen],'v' , - asfep->valuelen); + name = &asfep->nameval[0 ]; + value = &asfep->nameval[asfep->namelen]; + +if (want_obfuscate_pptr (asfep->flags, name, namelen, value, + asfep->valuelen)) { +obfuscate_parent_pointer (name, value, asfep->valuelen); + }else if (want_obfuscate_attr (asfep->flags, name, namelen, + value, asfep->valuelen)) { +generate_obfuscated_name (0 , asfep->namelen, name); +memset (value,'v' , asfep->valuelen); } asfep = (struct xfs_attr_sf_entry *)((char *)asfep +
@@ -1777,6 +2054,9 @@ process_attr_block(break ; }if (entry->flags &XFS_ATTR_LOCAL ) { +void *name, *value; +unsigned int valuelen; + local =xfs_attr3_leaf_name_local (leaf, i);if (local->namelen ==0 ) {if (show_warnings)
@@ -1785,11 +2065,21 @@ process_attr_block( (long long)cur_ino);break ; } -if (obfuscate) { + + name = &local->nameval[0 ]; + value = &local->nameval[local->namelen]; + valuelen =be16_to_cpu (local->valuelen); + +if (want_obfuscate_pptr (entry->flags, name, + local->namelen, value, + valuelen)) { +obfuscate_parent_pointer (name, value, valuelen); + }else if (want_obfuscate_attr (entry->flags, name, + local->namelen, value, + valuelen)) {generate_obfuscated_name (0 , local->namelen, - &local->nameval[0 ]); -memset (&local->nameval[local->namelen],'v' , -be16_to_cpu (local->valuelen)); + name); +memset (value,'v' , valuelen); }/* zero from end of nameval[] to next name start */ nlen = local->namelen;
@@ -3166,6 +3456,7 @@ metadump_f(pop_cur (); out:free (metablock); +remaptable_clear ();return 0 ; }
libxfs/libxfs_api_defs.h +2 -0
@@ -143,6 +143,8 @@#define xfs_parent_finish libxfs_parent_finish#define xfs_parent_irec_from_disk libxfs_parent_irec_from_disk#define xfs_parent_start libxfs_parent_start +#define xfs_parent_namecheck libxfs_parent_namecheck +#define xfs_parent_valuecheck libxfs_parent_valuecheck#define xfs_perag_get libxfs_perag_get#define xfs_perag_put libxfs_perag_put#define xfs_prealloc_blocks libxfs_prealloc_blocks
PATCH v11 26/32 xfs_db: hoist name obfuscation code out of metadump.c
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
We want to create a debugger command that will create obfuscated names for directory and xattr names, so hoist the name obfuscation code into a separate file.
--- db/Makefile | 2 db/metadump.c | 343 ------------------------------------------------------- db/obfuscate.c | 348 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ db/obfuscate.h | 16 +++ 4 files changed, 366 insertions(+), 343 deletions(-) create mode 100644 db/obfuscate.c create mode 100644 db/obfuscate.h
db/Makefile +1 -1
@@ -13,7 +13,7 @@ HFILES = addr.h agf.h agfl.h agi.h attr.h attrshort.h bit.h block.h bmap.h \ flist.h fprint.h frag.h freesp.h hash.h help.h init.h inode.h input.h \ io.h logformat.h malloc.h metadump.h output.h print.h quit.h sb.h \ sig.h strvec.h text.h type.h write.h attrset.h symlink.h fsmap.h \ - fuzz.h + fuzz.h obfuscate.h CFILES = $(HFILES:.h=.c) btdump.c btheight.c convert.c info.c namei.c \ timelimit.c LSRCFILES = xfs_admin.sh xfs_ncheck.sh xfs_metadump.sh
db/metadump.c +1 -342
361 lines not on this page · raw message
db/obfuscate.c new file +348 -0
348 lines not on this page · raw message
db/obfuscate.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2007, 2011 SGI +* All Rights Reserved. +*/ +#ifndef __DB_OBFUSCATE_H__ +#define __DB_OBFUSCATE_H__ + +/* Routines to obfuscate directory filenames and xattr names. */ + +#define is_invalid_char (c) ((c) == '/' || (c) == '\0') + +void obfuscate_name (xfs_dahash_t hash,size_t name_len,unsigned char *name); +int find_alternate (size_t name_len,unsigned char *name,uint32_t seq); + +#endif /* __DB_OBFUSCATE_H__ */
PATCH v11 27/32 xfs_db: create dirents and xattrs with colliding names
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new debugger command that will create dirent and xattr names that induce dahash collisions.
--- db/hash.c | 374 +++++++++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_db.8 | 31 ++++ 2 files changed, 405 insertions(+)
db/hash.c +374 -0
@@ -5,12 +5,15 @@*/ #include "libxfs.h" +#include "init.h" #include "addr.h" #include "command.h" #include "type.h" #include "io.h" #include "output.h" #include "hash.h" +#include "obfuscate.h" +#include <sys/xattr.h> static int hash_f (int argc,char **argv);static void hash_help (void );
@@ -46,8 +49,379 @@ hash_f(return 0 ; } +static void +hashcoll_help (void ) +{ +printf (_ ( +"\n" +" Generate obfuscated variants of the provided name. Each variant will have\n" +" the same dahash value. Names are written to stdout with a NULL separating\n" +" each name.\n" +"\n" +" -a -- create extended attributes.\n" +" -i -- read standard input for the name, up to %d bytes.\n" +" -n -- create this many names.\n" +" -p -- create directory entries or extended attributes in this file.\n" +" -s -- seed the rng with this value.\n" +"\n" ), +MAXNAMELEN -1 ); +} + +struct name_dup { +struct name_dup *next; +uint32_t crc; +uint8_t namelen; +uint8_t name[]; +}; + +static inline size_t +name_dup_sizeof ( +unsigned int namelen) +{ +return sizeof (struct name_dup ) + namelen; +} + +#define MAX_DUP_TABLE_BUCKETS (1048575) + +struct dup_table { +unsigned int nr_buckets; +struct name_dup *buckets[]; +}; + +static inline size_t +dup_table_sizeof ( +unsigned int nr_buckets) +{ +return sizeof (struct dup_table ) + + (nr_buckets *sizeof (struct name_dup *)); +} + +static int +dup_table_alloc ( +unsigned long nr_names, +struct dup_table **tabp) +{ +struct dup_table *t; + + *tabp =NULL ; + +if (nr_names ==1 ) +return 0 ; + + nr_names =min (MAX_DUP_TABLE_BUCKETS , nr_names); + t =calloc (1 ,dup_table_sizeof (nr_names)); +if (!t) +return ENOMEM ; + + t->nr_buckets = nr_names; + *tabp = t; +return 0 ; +} + +static void +dup_table_free ( +struct dup_table *tab) +{ +struct name_dup *ent, *next; +unsigned int i; + +if (!tab) +return ; + +for (i =0 ; i < tab->nr_buckets; i++) { + ent = tab->buckets[i]; + +while (ent) { + next = ent->next; +free (ent); + ent = next; + } + } +free (tab); +} + +static struct name_dup * +dup_table_find ( +struct dup_table *tab, +unsigned char *name, +size_t namelen) +{ +struct name_dup *ent; +uint32_t crc =crc32c (~0 , name, namelen); + + ent = tab->buckets[crc % tab->nr_buckets]; +while (ent) { +if (ent->crc == crc && + ent->namelen == namelen && + !memcmp (ent->name, name, namelen)) +return ent; + + ent = ent->next; + } + +return NULL ; +} + +static int +dup_table_store ( +struct dup_table *tab, +unsigned char *name, +size_t namelen) +{ +struct name_dup *dup; +uint32_t seq =1 ; + +ASSERT (namelen <MAXNAMELEN ); + +while ((dup =dup_table_find (tab, name, namelen)) !=NULL ) { +int ret; + +do { + ret =find_alternate (namelen, name, seq++); + }while (ret ==0 ); +if (ret <0 ) +return EEXIST ; + } + + dup =malloc (name_dup_sizeof (namelen)); +if (!dup) +return ENOMEM ; + + dup->crc =crc32c (~0 , name, namelen); + dup->namelen = namelen; +memcpy (dup->name, name, namelen); + dup->next = tab->buckets[dup->crc % tab->nr_buckets]; + + tab->buckets[dup->crc % tab->nr_buckets] = dup; +return 0 ; +} + +static int +collide_dirents ( +unsigned long nr, +const unsigned char *name, +size_t namelen, +int fd) +{ +struct xfs_name dname = { + .name = name, + .len = namelen, + }; +unsigned char direntname[MAXNAMELEN +1 ]; +struct dup_table *tab =NULL ; +xfs_dahash_t old_hash; +unsigned long i; +int error =0 ; + + old_hash =libxfs_dir2_hashname (mp, &dname); + +if (fd >=0 ) { +int newfd; + +/* +* User passed in a fd, so we'll use the directory to detect +* duplicate names. First create the name that we are passed +* in; the new names will be hardlinks to the first file. +*/ + newfd =openat (fd, name,O_CREAT ,0600 ); +if (newfd <0 ) +return errno; +close (newfd); + }else if (nr >1 ) { +/* +* Track every name we create so that we don't emit duplicates. +*/ + error =dup_table_alloc (nr, &tab); +if (error) +return error; + } + +for (i =0 ; i < nr; i++) { +strncpy (direntname, name,MAXNAMELEN ); +obfuscate_name (old_hash, namelen, direntname); + +if (fd >=0 ) { + error =linkat (fd, name, fd, direntname,0 ); +if (error && errno !=EEXIST ) +return errno; + +/* don't print names to stdout */ +continue ; + }else if (tab) { + error =dup_table_store (tab, direntname, namelen); +if (error) +break ; + } + +printf ("%s%c" , direntname,0 ); + } + +dup_table_free (tab); +return error; +} + +static int +collide_xattrs ( +unsigned long nr, +const unsigned char *name, +size_t namelen, +int fd) +{ +unsigned char xattrname[MAXNAMELEN +5 ]; +struct dup_table *tab =NULL ; +xfs_dahash_t old_hash; +unsigned long i; +int error; + + old_hash =libxfs_da_hashname (name, namelen); + +if (fd >=0 ) { +/* +* User passed in a fd, so we'll use the xattr structure to +* detect duplicate names. First create the attribute that we +* are passed in. +*/ +snprintf (xattrname,MAXNAMELEN +5 ,"user.%s" , name); + error =fsetxattr (fd, xattrname,"1" ,1 ,0 ); +if (error) +return errno; + }else if (nr >1 ) { +/* +* Track every name we create so that we don't emit duplicates. +*/ + error =dup_table_alloc (nr, &tab); +if (error) +return error; + } + +for (i =0 ; i < nr; i++) { +snprintf (xattrname,MAXNAMELEN +5 ,"user.%s" , name); +obfuscate_name (old_hash, namelen, xattrname +5 ); + +if (fd >=0 ) { + error =fsetxattr (fd, xattrname,"1" ,1 ,0 ); +if (error) +return errno; + +/* don't print names to stdout */ +continue ; + }else if (tab) { + error =dup_table_store (tab, xattrname, namelen +5 ); +if (error) +break ; + } + +printf ("%s%c" , xattrname,0 ); + } + +dup_table_free (tab); +return error; +} + +static int +hashcoll_f ( +int argc, +char **argv) +{ +const char *path =NULL ; +bool read_stdin = false; +bool create_xattr = false; +unsigned long nr =1 , seed =0 ; +int fd =-1 ; +int c; +int error; + +while ((c =getopt (argc, argv,"ain:p:s:" )) !=EOF ) { +switch (c) { +case 'a' : + create_xattr = true; +break ; +case 'i' : + read_stdin = true; +break ; +case 'n' : + nr =strtoul (optarg,NULL ,10 ); +break ; +case 'p' : + path = optarg; +break ; +case 's' : + seed =strtoul (optarg,NULL ,10 ); +break ; +default : + exitcode =1 ; +hashcoll_help (); +return 0 ; + } + } + +if (path) { +int oflags =O_RDWR ; + +if (!create_xattr) + oflags =O_RDONLY |O_DIRECTORY ; + + fd =open (path, oflags); +if (fd <0 ) { +perror (path); + exitcode =1 ; +return 0 ; + } + } + +if (seed) +srandom (seed); + +if (read_stdin) { +char buf[MAXNAMELEN ]; +size_t len; + + len =fread (buf,1 ,MAXNAMELEN -1 , stdin); + +if (create_xattr) + error =collide_xattrs (nr, buf, len, fd); +else + error =collide_dirents (nr, buf, len, fd); +if (error) { +printf (_ ("hashcoll: %s\n" ),strerror (error)); + exitcode =1 ; + } + goto done; + } + +for (c = optind; c < argc; c++) { +size_t len =strlen (argv[c]); + +if (create_xattr) + error =collide_xattrs (nr, argv[c], len, fd); +else + error =collide_dirents (nr, argv[c], len, fd); +if (error) { +printf (_ ("hashcoll: %s\n" ),strerror (error)); + exitcode =1 ; + } + } + +done: +if (fd >=0 ) +close (fd); +return 0 ; +} + +static cmdinfo_t hashcoll_cmd = { + .name ="hashcoll" , + .cfunc = hashcoll_f, + .argmin =0 , + .argmax =-1 , + .args =N_ ("[-a] [-s seed] [-n nr] [-p path] -i|names..." ), + .oneline =N_ ("create names that produce dahash collisions" ), + .help = hashcoll_help, +}; +void hash_init (void ) {add_command (&hash_cmd); +add_command (&hashcoll_cmd); } +
man/man8/xfs_db.8 +31 -0
@@ -768,6 +768,37 @@ Prints the hash value of .I string using the hash function of the XFS directory and attribute implementation. .TP +.BI "hashcoll [-a] [-s seed] [-n " nr "] [-p " path "] -i | " names... +Create directory entries or extended attributes names that all have the same +hash value. +The metadump name obfuscation algorithm is used here. +Names are written to standard output, with a NULL between each name for use +with xargs -0. +.RS 1.0i +.PD 0 +.TP 0.4i +.TP 0.4i +.B \-a +Create extended attribute names. +.TP 0.4i +.B \-i +Read the first name to create from standard input. +Up to 255 bytes are read. +If this option is not specified, first names are taken from the command line. +.TP 0.4i +.BI \-n " nr" +Create this many duplicated names. +The default is to create one name. +.TP 0.4i +.BI \-p " path" +Create directory entries or extended attributes in this file instead of +writing the names to standard output. +.TP 0.4i +.BI \-s " seed" +Seed the random number generator with this value. +.PD +.RE +.TP .BI "help [" command ] Print help for one or all commands. .TP
PATCH v11 28/32 xfs_db: add a parents command to list the parents of a file
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a command to dump the parents of a file.
--- db/namei.c | 335 +++++++++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_db.8 | 9 + 2 files changed, 344 insertions(+)
db/namei.c +335 -0
@@ -596,6 +596,338 @@ static struct cmdinfo ls_cmd = { .help = ls_help, }; +static void +pptr_emit ( +struct xfs_mount *mp, +const struct xfs_parent_name_irec *irec) +{ +struct xfs_name xname = { + .name = irec->p_name, + .len = irec->p_namelen, + }; +xfs_dahash_t hash; +bool good; + + hash =libxfs_dir2_hashname (mp, &xname); + good = hash == irec->p_namehash && +libxfs_dir2_namecheck (irec->p_name, irec->p_namelen); + +dbprintf ("%18llu:0x%08x 0x%08x:0x%08x %3d %.*s %s\n" , + irec->p_ino, irec->p_gen, irec->p_namehash, hash, + xname.len, xname.len, xname.name, + good ?_ ("(good)" ) :_ ("(corrupt)" )); +} + +static int +list_sf_pptrs ( +struct xfs_inode *ip) +{ +struct xfs_parent_name_irec irec; +struct xfs_attr_shortform *sf; +struct xfs_attr_sf_entry *sfe; +unsigned int i; + + sf = (struct xfs_attr_shortform *)ip->i_af.if_u1.if_data; +for (i =0 , sfe = &sf->list[0 ]; i < sf->hdr.count; i++) { +void *name = sfe->nameval; +void *value = &sfe->nameval[sfe->namelen]; + +if ((sfe->flags &XFS_ATTR_PARENT ) && +libxfs_parent_namecheck (mp, name, sfe->namelen, sfe->flags) && +libxfs_parent_valuecheck (mp, value, sfe->valuelen)) { +libxfs_parent_irec_from_disk (&irec, name, value, + sfe->valuelen); +pptr_emit (mp, &irec); + } + + sfe =xfs_attr_sf_nextentry (sfe); + } + +return 0 ; +} + +static void +list_leaf_pptr_entries ( +struct xfs_inode *ip, +struct xfs_buf *bp) +{ +struct xfs_parent_name_irec irec; +struct xfs_attr3_icleaf_hdr ichdr; +struct xfs_mount *mp = ip->i_mount; +struct xfs_attr_leafblock *leaf = bp->b_addr; +struct xfs_attr_leaf_entry *entry; +unsigned int i; + +libxfs_attr3_leaf_hdr_from_disk (mp->m_attr_geo, &ichdr, leaf); + entry =xfs_attr3_leaf_entryp (leaf); + +for (i =0 ; i < ichdr.count; entry++, i++) { +struct xfs_attr_leaf_name_local *name_loc; +void *value; +void *name; +unsigned int namelen, valuelen; + +if (!(entry->flags &XFS_ATTR_LOCAL ) || + !(entry->flags &XFS_ATTR_PARENT )) +continue ; + + name_loc =xfs_attr3_leaf_name_local (leaf, i); + name = name_loc->nameval; + namelen = name_loc->namelen; + value = &name_loc->nameval[name_loc->namelen]; + valuelen =be16_to_cpu (name_loc->valuelen); + +if (libxfs_parent_namecheck (mp, name, namelen, entry->flags) && +libxfs_parent_valuecheck (mp, value, valuelen)) { +libxfs_parent_irec_from_disk (&irec, name, value, + valuelen); +pptr_emit (mp, &irec); + } + } +} + +static int +list_leaf_pptrs ( +struct xfs_inode *ip) +{ +struct xfs_buf *leaf_bp; +int error; + + error = -libxfs_attr3_leaf_read (NULL , ip,0 , &leaf_bp); +if (error) +return error; + +list_leaf_pptr_entries (ip, leaf_bp); +libxfs_trans_brelse (NULL , leaf_bp); +return 0 ; +} + +static int +find_leftmost_attr_leaf ( +struct xfs_inode *ip, +struct xfs_buf **leaf_bpp) +{ +struct xfs_da3_icnode_hdr nodehdr; +struct xfs_mount *mp = ip->i_mount; +struct xfs_da_intnode *node; +struct xfs_da_node_entry *btree; +struct xfs_buf *bp; +xfs_dablk_t blkno =0 ; +unsigned int expected_level =0 ; +int error; + +for (;;) { +uint16_t magic; + + error = -libxfs_da3_node_read (NULL , ip, blkno, &bp, +XFS_ATTR_FORK ); +if (error) +return error; + + node = bp->b_addr; + magic =be16_to_cpu (node->hdr.info.magic); +if (magic ==XFS_ATTR_LEAF_MAGIC || + magic ==XFS_ATTR3_LEAF_MAGIC ) +break ; + + error =EFSCORRUPTED ; +if (magic !=XFS_DA_NODE_MAGIC && + magic !=XFS_DA3_NODE_MAGIC ) + goto out_buf; + +libxfs_da3_node_hdr_from_disk (mp, &nodehdr, node); + +if (nodehdr.count ==0 || nodehdr.level >=XFS_DA_NODE_MAXDEPTH ) + goto out_buf; + +/* Check the level from the root node. */ +if (blkno ==0 ) + expected_level = nodehdr.level -1 ; +else if (expected_level != nodehdr.level) + goto out_buf; +else + expected_level--; + +/* Find the next level towards the leaves of the dabtree. */ + btree = nodehdr.btree; + blkno =be32_to_cpu (btree->before); +libxfs_trans_brelse (NULL , bp); + } + + error =EFSCORRUPTED ; +if (expected_level !=0 ) + goto out_buf; + + *leaf_bpp = bp; +return 0 ; + +out_buf: +libxfs_trans_brelse (NULL , bp); +return error; +} + +static int +list_node_pptrs ( +struct xfs_inode *ip) +{ +struct xfs_attr3_icleaf_hdr leafhdr; +struct xfs_mount *mp = ip->i_mount; +struct xfs_attr_leafblock *leaf; +struct xfs_buf *leaf_bp; +int error; + + error =find_leftmost_attr_leaf (ip, &leaf_bp); +if (error) +return error; + +for (;;) { +list_leaf_pptr_entries (ip, leaf_bp); + +/* Find the right sibling of this leaf block. */ + leaf = leaf_bp->b_addr; +libxfs_attr3_leaf_hdr_from_disk (mp->m_attr_geo, &leafhdr, leaf); +if (leafhdr.forw ==0 ) + goto out_leaf; + +libxfs_trans_brelse (NULL , leaf_bp); + + error = -libxfs_attr3_leaf_read (NULL , ip, leafhdr.forw, + &leaf_bp); +if (error) +return error; + } + +out_leaf: +libxfs_trans_brelse (NULL , leaf_bp); +return error; +} + +static int +list_pptrs ( +struct xfs_inode *ip) +{ +int error; + +if (!libxfs_inode_hasattr (ip)) +return 0 ; + +if (ip->i_af.if_format ==XFS_DINODE_FMT_LOCAL ) +return list_sf_pptrs (ip); + +/* attr functions require that the attr fork is loaded */ + error = -libxfs_iread_extents (NULL , ip,XFS_ATTR_FORK ); +if (error) +return error; + +if (libxfs_attr_is_leaf (ip)) +return list_leaf_pptrs (ip); + +return list_node_pptrs (ip); +} + +/* If the io cursor points to a file, list its parents. */ +static int +parent_cur ( +char *tag) +{ +struct xfs_inode *ip; +int error =0 ; + +if (!xfs_has_parent (mp)) +return 0 ; + +if (iocur_top->typ != &typtab[TYP_INODE ]) +return ENOTDIR ; + + error = -libxfs_iget (mp,NULL , iocur_top->ino,0 , &ip); +if (error) +return error; + +/* List the parents of a file. */ +if (tag) +dbprintf (_ ("%s:\n" ), tag); + + error =list_pptrs (ip); +if (error) + goto rele; + +rele: +libxfs_irele (ip); +return error; +} + +static void +parent_help (void ) +{ +dbprintf (_ ( +"\n" +" List the parents of the currently selected file.\n" +"\n" +" Parent pointers will be listed in the format:\n" +" inode_number:inode_gen ondisk_namehash:namehash name_length name\n" + )); +} + +static int +parent_f ( +int argc, +char **argv) +{ +int c; +int error =0 ; + +while ((c =getopt (argc, argv,"" )) !=-1 ) { +switch (c) { +default : +ls_help (); +return 0 ; + } + } + +if (optind == argc) { + error =parent_cur (NULL ); +if (error) { +dbprintf ("%s\n" ,strerror (error)); + exitcode =1 ; + } + +return 0 ; + } + +for (c = optind; c < argc; c++) { +push_cur (); + + error =path_walk (argv[c]); +if (error) + goto err_cur; + + error =parent_cur (argv[c]); +if (error) + goto err_cur; + +pop_cur (); + } + +return 0 ; +err_cur: +pop_cur (); +if (error) { +dbprintf ("%s: %s\n" , argv[c],strerror (error)); + exitcode =1 ; + } +return 0 ; +} + +static struct cmdinfo parent_cmd = { + .name ="parent" , + .altname ="pptr" , + .cfunc = parent_f, + .argmin =0 , + .argmax =-1 , + .canpush =0 , + .args ="[paths...]" , + .help = parent_help, +};void namei_init (void ) {
@@ -604,4 +936,7 @@ namei_init(void) ls_cmd.oneline =_ ("list directory contents" );add_command (&ls_cmd); + + parent_cmd.oneline =_ ("list parent pointers" ); +add_command (&parent_cmd); }
man/man8/xfs_db.8 +9 -0
@@ -884,6 +884,15 @@ See the .B print command. .TP +.BI "parent [" paths "]..." +List the parents of a file. +If a path resolves to a file, the parents of that file will be listed. +If no paths are supplied and the IO cursor points at an inode, the parents of +that file will be listed. + +The output format is: +inode number, inode generation, ondisk namehash, namehash, name length, name. +.TP .BI "path " dir_path Walk the directory tree to an inode using the supplied path. Absolute and relative paths are supported.
PATCH v11 29/32 libxfs: create new files with attr forks if necessary
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create new files with attr forks if they're going to have parent pointers. In the next patch we'll fix mkfs to use the same parent creation functions as the kernel, so we're going to need this.
--- libxfs/init.c | 4 ++++ libxfs/util.c | 14 ++++++++++++++ 2 files changed, 18 insertions(+)
libxfs/init.c +4 -0
@@ -746,14 +746,18 @@ voidlibxfs_compute_all_maxlevels ( struct xfs_mount *mp) { +struct xfs_ino_geometry *igeo =M_IGEO (mp); +xfs_alloc_compute_maxlevels (mp);xfs_bmap_compute_maxlevels (mp,XFS_DATA_FORK );xfs_bmap_compute_maxlevels (mp,XFS_ATTR_FORK ); + igeo->attr_fork_offset =xfs_bmap_compute_attr_offset (mp);xfs_ialloc_setup_geometry (mp);xfs_rmapbt_compute_maxlevels (mp);xfs_refcountbt_compute_maxlevels (mp);xfs_agbtree_compute_maxlevels (mp); + }/*
libxfs/util.c +14 -0
@@ -322,6 +322,20 @@ libxfs_init_new_inode(ASSERT (0 ); } +/* +* If we need to create attributes immediately after allocating the +* inode, initialise an empty attribute fork right now. We use the +* default fork offset for attributes here as we don't know exactly what +* size or how many attributes we might be adding. We can do this +* safely here because we know the data fork is completely empty and +* this saves us from needing to run a separate transaction to set the +* fork offset in the immediate future. +*/ +if (xfs_has_parent (tp->t_mountp) &&xfs_has_attr (tp->t_mountp)) { + ip->i_forkoff =xfs_default_attroffset (ip) >>3 ; +xfs_ifork_init_attr (ip,XFS_DINODE_FMT_EXTENTS ,0 ); + } +/* * Log the new values stuffed into the inode. */
PATCH v11 30/32 xfsprogs: Fix default superblock attr bits
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Recent parent pointer testing discovered that the default attr configuration has XFS_SB_VERSION2_ATTR2BIT enabled but XFS_SB_VERSION_ATTRBIT disabled. This is incorrect since XFS_SB_VERSION2_ATTR2BIT describes the format of the attr where as XFS_SB_VERSION_ATTRBIT enables or disables attrs. Fix this by enableing XFS_SB_VERSION_ATTRBIT for either attr version 1 or 2
--- mkfs/xfs_mkfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mkfs/xfs_mkfs.c +1 -1
@@ -3266,7 +3266,7 @@ sb_set_features( sbp->sb_versionnum |=XFS_SB_VERSION_DALIGNBIT ;if (fp->log_version ==2 ) sbp->sb_versionnum |=XFS_SB_VERSION_LOGV2BIT ; -if (fp->attr_version ==1 ) +if (fp->attr_version >=1 ) sbp->sb_versionnum |=XFS_SB_VERSION_ATTRBIT ;if (fp->nci) sbp->sb_versionnum |=XFS_SB_VERSION_BORGBIT ;
PATCH v11 31/32 mkfs: Add parent pointers during protofile creation
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Inodes created from protofile parsing will also need to add the appropriate parent pointers.
--- mkfs/proto.c | 48 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-)
mkfs/proto.c +47 -1
@@ -378,6 +378,20 @@ newdirectory(fail (_ ("directory create error" ), error); } +static struct xfs_parent_defer * +newpptr ( +struct xfs_mount *mp) +{ +struct xfs_parent_defer *ret; +int error; + + error = -libxfs_parent_start (mp, &ret); +if (error) +fail (_ ("initializing parent pointer" ), error); + +return ret; +} + static void parseproto( xfs_mount_t *mp,
@@ -412,6 +426,7 @@ parseproto(struct cred creds;char *value;struct xfs_name xname; +struct xfs_parent_defer *parent =NULL ;memset (&creds,0 ,sizeof (creds)); mstr =getstr (pp);
@@ -486,6 +501,7 @@ parseproto(case IF_REGULAR : buf =newregfile (pp, &len); tp =getres (mp,XFS_B_TO_FSB (mp, len)); + parent =newpptr (mp); error = -libxfs_dir_ialloc (&tp, pip, mode|S_IFREG ,1 ,0 , &creds, fsxp, &ip);if (error)
@@ -509,7 +525,7 @@ parseproto(exit (1 ); } tp =getres (mp,XFS_B_TO_FSB (mp, llen)); - + parent =newpptr (mp); error = -libxfs_dir_ialloc (&tp, pip, mode|S_IFREG ,1 ,0 , &creds, fsxp, &ip);if (error)
@@ -520,15 +536,24 @@ parseproto( xname.type =XFS_DIR3_FT_REG_FILE ;newdirent (mp, tp, pip, &xname, ip->i_ino);libxfs_trans_log_inode (tp, ip, flags); +if (parent) { + error = -libxfs_parent_add (tp, parent, pip, &xname, + ip); +if (error) +fail (_ ("committing parent pointers failed." ), + error); + } error = -libxfs_trans_commit (tp);if (error)fail (_ ("Space preallocation failed." ), error); +libxfs_parent_finish (mp, parent);rsvfile (mp, ip, llen);libxfs_irele (ip);return ;case IF_BLOCK : tp =getres (mp,0 ); + parent =newpptr (mp); majdev =getnum (getstr (pp),0 ,0 , false); mindev =getnum (getstr (pp),0 ,0 , false); error = -libxfs_dir_ialloc(&tp, pip, mode|S_IFBLK, 1,
@@ -544,6 +569,7 @@ parseproto(case IF_CHAR : tp =getres (mp,0 ); + parent =newpptr (mp); majdev =getnum (getstr (pp),0 ,0 , false); mindev =getnum (getstr (pp),0 ,0 , false); error = -libxfs_dir_ialloc(&tp, pip, mode|S_IFCHR, 1,
@@ -558,6 +584,7 @@ parseproto(case IF_FIFO : tp =getres (mp,0 ); + parent =newpptr (mp); error = -libxfs_dir_ialloc (&tp, pip, mode|S_IFIFO ,1 ,0 , &creds, fsxp, &ip);if (error)
@@ -570,6 +597,7 @@ parseproto( buf =getstr (pp); len = (int )strlen (buf); tp =getres (mp,XFS_B_TO_FSB (mp, len)); + parent =newpptr (mp); error = -libxfs_dir_ialloc (&tp, pip, mode|S_IFLNK ,1 ,0 , &creds, fsxp, &ip);if (error)
@@ -592,6 +620,7 @@ parseproto(libxfs_log_sb (tp); isroot =1 ; }else { + parent =newpptr (mp);libxfs_trans_ijoin (tp, pip,0 ); xname.type =XFS_DIR3_FT_DIR ;newdirent (mp, tp, pip, &xname, ip->i_ino);
@@ -600,9 +629,19 @@ parseproto( }newdirectory (mp, tp, ip, pip);libxfs_trans_log_inode (tp, ip, flags); +if (parent) { + error = -libxfs_parent_add (tp, parent, pip, &xname, + ip); +if (error) +fail (_ ("committing parent pointers failed." ), + error); + } error = -libxfs_trans_commit (tp);if (error)fail (_ ("Directory inode allocation failed." ), error); + +libxfs_parent_finish (mp, parent); +/* * RT initialization. Do this here to ensure that * the RT inodes get placed after the root inode.
@@ -625,11 +664,18 @@ parseproto(fail (_ ("Unknown format" ),EINVAL ); }libxfs_trans_log_inode (tp, ip, flags); +if (parent) { + error = -libxfs_parent_add (tp, parent, pip, &xname, ip); +if (error) +fail (_ ("committing parent pointers failed." ), error); + } error = -libxfs_trans_commit (tp);if (error) {fail (_ ("Error encountered creating file from prototype file" ), error); } + +libxfs_parent_finish (mp, parent);libxfs_irele (ip); }
PATCH v11 32/32 mkfs: enable formatting with parent pointers
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Enable parent pointer support in mkfs via the '-n parent' parameter.
--- mkfs/xfs_mkfs.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)
mkfs/xfs_mkfs.c +26 -3
@@ -110,6 +110,7 @@ enum {N_SIZE =0 ,N_VERSION ,N_FTYPE , +N_PARENT , N_MAX_OPTS, };
@@ -621,6 +622,7 @@ static struct opt_params nopts = { [N_SIZE] = "size", [N_VERSION] = "version", [N_FTYPE] = "ftype", + [N_PARENT] = "parent", [N_MAX_OPTS ] =NULL , }, .subopt_params = {
@@ -644,6 +646,14 @@ static struct opt_params nopts = { .maxval = 1, .defaultval = 1, }, + { .index = N_PARENT, + .conflicts = { { NULL, LAST_CONFLICT } }, + .minval = 0, + .maxval = 1, + .defaultval = 1, + }, + + }, };
@@ -1000,7 +1010,7 @@ usage( void ) /* log subvol */ [-l agnum=n,internal,size=num,logdev=xxx,version=n\n\ sunit=value|su=num,sectsize=num,lazy-count=0|1]\n\ /* label */ [-L label (maximum 12 characters)]\n\ -/* naming */ [-n size=num,version=2|ci,ftype=0|1]\n\ +/* naming */ [-n size=num,version=2|ci,ftype=0|1,parent=0|1]]\n\ /* no-op info only */ [-N]\n\ /* prototype file */ [-p fname]\n\ /* quiet */ [-q]\n\
@@ -1774,6 +1784,9 @@ naming_opts_parser(case N_FTYPE : cli->sb_feat.dirftype =getnum (value, opts, subopt);break ; +case N_PARENT : + cli->sb_feat.parent_pointers =getnum (value, &nopts,N_PARENT ); +break ;default :return -EINVAL ; }
@@ -2286,6 +2299,14 @@ _("inode btree counters not supported without finobt support\n")); cli->sb_feat.inobtcnt = false; } +if ((cli->sb_feat.parent_pointers) && + cli->sb_feat.dir_version ==4 ) { +fprintf (stderr, +_ ("parent pointers not supported on v4 filesystems\n" )); +usage (); + cli->sb_feat.parent_pointers = false; + } +if (cli->xi->rtname) {if (cli->sb_feat.reflink &&cli_opt_set (&mopts,M_REFLINK )) { fprintf(stderr,
@@ -3285,8 +3306,6 @@ sb_set_features( sbp->sb_features2 |=XFS_SB_VERSION2_LAZYSBCOUNTBIT ;if (fp->projid32bit) sbp->sb_features2 |=XFS_SB_VERSION2_PROJID32BIT ; -if (fp->parent_pointers) - sbp->sb_features2 |=XFS_SB_VERSION2_PARENTBIT ;if (fp->crcs_enabled) sbp->sb_features2 |=XFS_SB_VERSION2_CRCBIT ;if (fp->attr_version ==2 )
@@ -3327,6 +3346,10 @@ sb_set_features( sbp->sb_features_ro_compat |=XFS_SB_FEAT_RO_COMPAT_INOBTCNT ;if (fp->bigtime) sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_BIGTIME ; +if (fp->parent_pointers) { + sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_PARENT ; + sbp->sb_versionnum |=XFS_SB_VERSION_ATTRBIT ; + }/* * Sparse inode chunk support has two main inode alignment requirements.
v11 cover xfs_repair: support parent pointers
In reply to Darrick J. WongHi all,
This patchset implements offline checking and repair for parent pointers. We do this rather expensively by constructing a (per-AG) master list of parent pointers for inodes rooted in that AG. Next, we walk each inode of that AG, construct an index of that file's parent pointers, and then compare the file index against the relevant part of the master index. From there we can sync the parent pointers as needed.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-offline-repair
--- libxfs/libxfs_api_defs.h | 7 libxfs/xfblob.c | 9 libxfs/xfblob.h | 2 repair/Makefile | 6 repair/listxattr.c | 271 ++++++++++ repair/listxattr.h | 15 + repair/phase6.c | 41 + repair/pptr.c | 1271 ++++++++++++++++++++++++++++++++++++++++++++++ repair/pptr.h | 17 + repair/strblobs.c | 212 ++++++++ repair/strblobs.h | 24 + 11 files changed, 1873 insertions(+), 2 deletions(-) create mode 100644 repair/listxattr.c create mode 100644 repair/listxattr.h create mode 100644 repair/pptr.c create mode 100644 repair/pptr.h create mode 100644 repair/strblobs.c create mode 100644 repair/strblobs.h
PATCH v11 1/7 xfs_repair: build a parent pointer index
From: Darrick J. Wong <[email protected]>
When we're walking directories during phase 6, build an index of parent pointers that we expect to find.
--- repair/Makefile | 2 + repair/phase6.c | 39 ++++++++++- repair/pptr.c | 198 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ repair/pptr.h | 15 ++++ 4 files changed, 252 insertions(+), 2 deletions(-) create mode 100644 repair/pptr.c create mode 100644 repair/pptr.h
repair/Makefile +2 -0
@@ -23,6 +23,7 @@ HFILES = \ err_protos.h \ globals.h \ incore.h \ + pptr.h \ prefetch.h \ progress.h \ protos.h \
@@ -59,6 +60,7 @@ CFILES = \ phase5.c \ phase6.c \ phase7.c \ + pptr.c \ prefetch.c \ progress.c \ quotacheck.c \
repair/phase6.c +37 -2
@@ -18,6 +18,7 @@#include "dinode.h" #include "progress.h" #include "versions.h" +#include "repair/pptr.h" static struct cred zerocr;static struct fsxattr zerofsx;
@@ -974,6 +975,7 @@ mk_orphanage(xfs_mount_t *mp)do_error (_ ("can't make %s, createname error %d\n" ),ORPHANAGE , error); +add_parent_ptr (ip->i_ino,ORPHANAGE , pip);/* * bump up the link count in the root directory to account
@@ -1160,6 +1162,10 @@ mv_orphanage(do_error (_ ("orphanage name create failed (%d)\n" ), err); } + +if (xfs_has_parent (mp)) +add_parent_ptr (ino_p->i_ino, xname.name, orphanage_ip); +libxfs_irele (ino_p);libxfs_irele (orphanage_ip); }
@@ -2459,6 +2465,7 @@ shortform_dir2_entry_check(struct xfs_dir2_sf_entry *next_sfep;struct xfs_ifork *ifp;struct ino_tree_node *irec; +xfs_dir2_dataptr_t diroffset;int max_size;int ino_offset;int i;
@@ -2637,8 +2644,9 @@ shortform_dir2_entry_check(/* * check for duplicate names in directory. */ -if (!dir_hash_add (mp, hashtab, (xfs_dir2_dataptr_t) - (sfep -xfs_dir2_sf_firstentry (sfp)), + diroffset =xfs_dir2_byte_to_dataptr ( +xfs_dir2_sf_get_offset (sfep)); +if (!dir_hash_add (mp, hashtab, diroffset, lino, sfep->namelen, sfep->name,libxfs_dir2_sf_get_ftype (mp, sfep))) { do_warn(
@@ -2672,6 +2680,7 @@ _("entry \"%s\" (ino %" PRIu64 ") in dir %" PRIu64 " is a duplicate name"), next_sfep =shortform_dir2_junk (mp, sfp, sfep, lino, &max_size, &i, &bytes_deleted, ino_dirty); +dir_hash_junkit (hashtab, diroffset);continue ; }else if (parent == ino) {add_inode_reached (irec, ino_offset);
@@ -2696,6 +2705,7 @@ _("entry \"%s\" (ino %" PRIu64 ") in dir %" PRIu64 " is a duplicate name"), next_sfep =shortform_dir2_junk (mp, sfp, sfep, lino, &max_size, &i, &bytes_deleted, ino_dirty); +dir_hash_junkit (hashtab, diroffset);continue ; } }
@@ -2787,6 +2797,26 @@ _("entry \"%s\" (ino %" PRIu64 ") in dir %" PRIu64 " is a duplicate name"), } } +static void +dir_hash_add_parent_ptrs ( +struct xfs_inode *dp, +struct dir_hash_tab *hashtab) +{ +struct dir_hash_ent *p; + +if (!xfs_has_parent (dp->i_mount)) +return ; + +for (p = hashtab->first; p; p = p->nextbyorder) { +if (p->name.name[0 ] =='/' || (p->name.name[0 ] =='.' && + (p->name.len ==1 || (p->name.len ==2 && + p->name.name[1 ] =='.' )))) +continue ; + +add_parent_ptr (p->inum, p->name.name, dp); + } +} +/* * processes all reachable inodes in directories */
@@ -2913,6 +2943,7 @@ _("error %d fixing shortform directory %llu\n"),default :break ; } +dir_hash_add_parent_ptrs (ip, hashtab);dir_hash_done (hashtab);/*
@@ -3204,6 +3235,8 @@ phase6(xfs_mount_t *mp)ino_tree_node_t *irec;int i; +parent_ptr_init (mp); +memset (&zerocr,0 ,sizeof (struct cred ));memset (&zerofsx,0 ,sizeof (struct fsxattr )); orphanage_ino =0 ;
@@ -3304,4 +3337,6 @@ _(" - resetting contents of realtime bitmap and summary inodes\n")); irec =next_ino_rec (irec); } } + +parent_ptr_free (mp); }
repair/pptr.c new file +198 -0
198 lines not on this page · raw message
repair/pptr.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __REPAIR_PPTR_H__ +#define __REPAIR_PPTR_H__ + +void parent_ptr_free (struct xfs_mount *mp); +void parent_ptr_init (struct xfs_mount *mp); + +void add_parent_ptr (xfs_ino_t ino,const unsigned char *fname, +struct xfs_inode *dp); + +#endif /* __REPAIR_PPTR_H__ */
PATCH v11 2/7 xfs_repair: move the global dirent name store to a separate object
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Abstract the main parent pointer dirent names xfblob object into a separate data structure to hide implementation details.
The goals here are (a) reduce memory usage when we can by deduplicating dirent names that exist in multiple directories; and (b) provide a unique id for each name in the system so that sorting incore parent pointer records can be done in a stable manner. Fast stable sorting of records is required for the dirent <-> pptr matching algorithm.
--- repair/Makefile | 2 + repair/pptr.c | 11 ++++--- repair/strblobs.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++++++ repair/strblobs.h | 20 +++++++++++++ 4 files changed, 108 insertions(+), 5 deletions(-) create mode 100644 repair/strblobs.c create mode 100644 repair/strblobs.h
repair/Makefile +2 -0
@@ -32,6 +32,7 @@ HFILES = \ rt.h \ scan.h \ slab.h \ + strblobs.h \ threads.h \ versions.h
@@ -69,6 +70,7 @@ CFILES = \ sb.c \ scan.c \ slab.c \ + strblobs.c \ threads.c \ versions.c \ xfs_repair.c
repair/pptr.c +6 -5
@@ -9,6 +9,7 @@#include "repair/err_protos.h" #include "repair/slab.h" #include "repair/pptr.h" +#include "repair/strblobs.h" #undef PPTR_DEBUG
@@ -55,7 +56,7 @@* This tuple is recorded in the per-AG master parent pointer index. Note * that names are stored separately in an xfblob data structure so that the * rest of the information can be sorted and processed as fixed-size records; -* the incore parent pointer record contains a pointer to the xfblob data. +* the incore parent pointer record contains a pointer to the strblob data. */ struct ag_pptr {
@@ -85,7 +86,7 @@ struct ag_pptrs { };/* Global names storage file. */ -static struct xfblob *names; +static struct strblobs *nameblobs;static pthread_mutex_t names_mutex =PTHREAD_MUTEX_INITIALIZER ;static struct ag_pptrs *fs_pptrs;
@@ -105,7 +106,7 @@ parent_ptr_free(free (fs_pptrs); fs_pptrs =NULL ; -xfblob_destroy (names); +strblobs_destroy (&nameblobs); }void
@@ -118,7 +119,7 @@ parent_ptr_init(if (!xfs_has_parent (mp))return ; - error = -xfblob_create (mp,"parent pointer names" , &names); + error =strblobs_init (mp,"parent pointer names" , &nameblobs);if (error)do_error (_ ("init parent pointer names failed: %s\n" ),strerror (error));
@@ -173,7 +174,7 @@ add_parent_ptr( ag_pptr.namehash =libxfs_dir2_hashname (mp, &dname);pthread_mutex_lock (&names_mutex); - error = -xfblob_store (names, &ag_pptr.name_cookie, fname, + error =strblobs_store (nameblobs, &ag_pptr.name_cookie, fname, ag_pptr.namelen);pthread_mutex_unlock (&names_mutex);if (error)
repair/strblobs.c new file +80 -0
80 lines not on this page · raw message
repair/strblobs.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __REPAIR_STRBLOBS_H__ +#define __REPAIR_STRBLOBS_H__ + +struct strblobs ; + +int strblobs_init (struct xfs_mount *mp,const char *descr, +struct strblobs **sblobs); +void strblobs_destroy (struct strblobs **sblobs); + +int strblobs_store (struct strblobs *sblobs,xfblob_cookie *str_cookie, +const unsigned char *str,unsigned int str_len); +int strblobs_load (struct strblobs *sblobs,xfblob_cookie str_cookie, +unsigned char *str,unsigned int str_len); + +#endif /* __REPAIR_STRBLOBS_H__ */
PATCH v11 3/7 xfs_repair: deduplicate strings stored in string blob
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Reduce the memory requirements of the string blob structure by deduplicating the strings stored within.
--- repair/pptr.c | 13 ++++- repair/strblobs.c | 140 +++++++++++++++++++++++++++++++++++++++++++++++++++-- repair/strblobs.h | 8 ++- 3 files changed, 152 insertions(+), 9 deletions(-)
repair/pptr.c +10 -3
@@ -49,7 +49,7 @@* * becomes this backwards information: * -* (child_agino*, dir_ino*, dir_gen, name*) +* (child_agino*, dir_ino*, dir_gen, name_cookie*) * * Key fields are starred. *
@@ -57,6 +57,10 @@* that names are stored separately in an xfblob data structure so that the * rest of the information can be sorted and processed as fixed-size records; * the incore parent pointer record contains a pointer to the strblob data. +* Because string blobs are deduplicated, there's a 1:1 mapping of name cookies +* to strings, which means that we can use the name cookie as a comparison key +* instead of loading the full dentry name every time we want to perform a +* comparison. */ struct ag_pptr {
@@ -113,13 +117,16 @@ voidparent_ptr_init ( struct xfs_mount *mp) { +uint64_t iused;xfs_agnumber_t agno;int error;if (!xfs_has_parent (mp))return ; - error =strblobs_init (mp,"parent pointer names" , &nameblobs); +/* One hash bucket per inode, up to about 8M of memory on 64-bit. */ + iused =min (mp->m_sb.sb_icount - mp->m_sb.sb_ifree,1048573 ); + error =strblobs_init (mp,"parent pointer names" , iused, &nameblobs);if (error)do_error (_ ("init parent pointer names failed: %s\n" ),strerror (error));
@@ -175,7 +182,7 @@ add_parent_ptr(pthread_mutex_lock (&names_mutex); error =strblobs_store (nameblobs, &ag_pptr.name_cookie, fname, - ag_pptr.namelen); + ag_pptr.namelen, ag_pptr.namehash);pthread_mutex_unlock (&names_mutex);if (error) do_error(_("storing name '%s' failed: %s\n"),
repair/strblobs.c +136 -4
186 lines not on this page · raw message
repair/strblobs.h +6 -2
@@ -9,12 +9,16 @@struct strblobs ;int strblobs_init (struct xfs_mount *mp,const char *descr, -struct strblobs **sblobs); +unsigned int hash_buckets,struct strblobs **sblobs);void strblobs_destroy (struct strblobs **sblobs);int strblobs_store (struct strblobs *sblobs,xfblob_cookie *str_cookie, -const unsigned char *str,unsigned int str_len); +const unsigned char *str,unsigned int str_len, +xfs_dahash_t hash);int strblobs_load (struct strblobs *sblobs,xfblob_cookie str_cookie,unsigned char *str,unsigned int str_len); +int strblobs_lookup (struct strblobs *sblobs,xfblob_cookie *str_cookie, +const unsigned char *str,unsigned int str_len, +xfs_dahash_t hash);#endif /* __REPAIR_STRBLOBS_H__ */
PATCH v11 4/7 xfs_repair: check parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Use the parent pointer index that we constructed in the previous patch to check that each file's parent pointer records exactly match the directory entries that we recorded while walking directory entries.
--- libxfs/libxfs_api_defs.h | 4 libxfs/xfblob.c | 9 + libxfs/xfblob.h | 2 repair/Makefile | 2 repair/listxattr.c | 271 +++++++++++++++ repair/listxattr.h | 15 + repair/phase6.c | 2 repair/pptr.c | 813 ++++++++++++++++++++++++++++++++++++++++++++++ repair/pptr.h | 2 9 files changed, 1120 insertions(+) create mode 100644 repair/listxattr.c create mode 100644 repair/listxattr.h
libxfs/libxfs_api_defs.h +4 -0
@@ -34,7 +34,9 @@#define xfs_alloc_vextent libxfs_alloc_vextent#define xfs_attr3_leaf_hdr_from_disk libxfs_attr3_leaf_hdr_from_disk +#define xfs_attr3_leaf_read libxfs_attr3_leaf_read#define xfs_attr_get libxfs_attr_get +#define xfs_attr_is_leaf libxfs_attr_is_leaf#define xfs_attr_leaf_newentsize libxfs_attr_leaf_newentsize#define xfs_attr_namecheck libxfs_attr_namecheck#define xfs_attr_set libxfs_attr_set
@@ -63,6 +65,7 @@#define xfs_bwrite libxfs_bwrite#define xfs_calc_dquots_per_chunk libxfs_calc_dquots_per_chunk#define xfs_da3_node_hdr_from_disk libxfs_da3_node_hdr_from_disk +#define xfs_da3_node_read libxfs_da3_node_read#define xfs_da_get_buf libxfs_da_get_buf#define xfs_da_hashname libxfs_da_hashname#define xfs_da_read_buf libxfs_da_read_buf
@@ -130,6 +133,7 @@#define xfs_inobt_stage_cursor libxfs_inobt_stage_cursor#define xfs_inode_from_disk libxfs_inode_from_disk#define xfs_inode_from_disk_ts libxfs_inode_from_disk_ts +#define xfs_inode_hasattr libxfs_inode_hasattr#define xfs_inode_to_disk libxfs_inode_to_disk#define xfs_inode_validate_cowextsize libxfs_inode_validate_cowextsize#define xfs_inode_validate_extsize libxfs_inode_validate_extsize
libxfs/xfblob.c +9 -0
@@ -146,3 +146,12 @@ xfblob_free(xfile_discard (blob->xfile, cookie,sizeof (key) + key.xb_size);return 0 ; } + +/* Drop all the blobs. */ +void +xfblob_truncate ( +struct xfblob *blob) +{ +xfile_discard (blob->xfile,PAGE_SIZE , blob->last_offset -PAGE_SIZE ); + blob->last_offset =PAGE_SIZE ; +}
libxfs/xfblob.h +2 -0
@@ -22,4 +22,6 @@ int xfblob_store(struct xfblob *blob, xfblob_cookie *cookie, const void *ptr, uint32_t size);int xfblob_free (struct xfblob *blob,xfblob_cookie cookie); +void xfblob_truncate (struct xfblob *blob); +#endif /* __XFS_SCRUB_XFBLOB_H__ */
repair/Makefile +2 -0
@@ -23,6 +23,7 @@ HFILES = \ err_protos.h \ globals.h \ incore.h \ + listxattr.h \ pptr.h \ prefetch.h \ progress.h \
@@ -54,6 +55,7 @@ CFILES = \ incore_ext.c \ incore_ino.c \ init.c \ + listxattr.c \ phase1.c \ phase2.c \ phase3.c \
repair/listxattr.c new file +271 -0
271 lines not on this page · raw message
repair/listxattr.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2022-2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __REPAIR_LISTXATTR_H__ +#define __REPAIR_LISTXATTR_H__ + +typedef int (*xattr_walk_fn )(struct xfs_inode *ip,unsigned int attr_flags, +const unsigned char *name,unsigned int namelen, +const void *value,unsigned int valuelen,void *priv); + +int xattr_walk (struct xfs_inode *ip,xattr_walk_fn attr_fn,void *priv); + +#endif /* __REPAIR_LISTXATTR_H__ */
repair/phase6.c +2 -0
@@ -3338,5 +3338,7 @@ _(" - resetting contents of realtime bitmap and summary inodes\n")); } } +/* Check and repair directory parent pointers, if enabled. */ +check_parent_ptrs (mp);parent_ptr_free (mp); }
repair/pptr.c +813 -0
847 lines not on this page · raw message
repair/pptr.h +2 -0
@@ -12,4 +12,6 @@ void parent_ptr_init(struct xfs_mount *mp);void add_parent_ptr (xfs_ino_t ino,const unsigned char *fname,struct xfs_inode *dp); +void check_parent_ptrs (struct xfs_mount *mp); +#endif /* __REPAIR_PPTR_H__ */
PATCH v11 5/7 xfs_repair: dump garbage parent pointer attributes
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Delete xattrs that have ATTR_PARENT set but are so garbage that they clearly aren't parent pointers.
--- repair/pptr.c | 142 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 140 insertions(+), 2 deletions(-)
repair/pptr.c +140 -2
@@ -191,6 +191,29 @@ struct file_scan {/* Does this file have garbage xattrs with ATTR_PARENT set? */ bool have_garbage; + +/* xattrs that we have to remove from this file */ +struct xfs_slab *garbage_xattr_recs; + +/* attr names associated with garbage_xattr_recs */ +struct xfblob *garbage_xattr_names; +}; + +struct garbage_xattr { +/* xfs_da_args.attr_filter for the attribute being removed */ +unsigned int attr_filter; + +/* attribute name length */ +unsigned int attrnamelen; + +/* attribute value length */ +unsigned int attrvaluelen; + +/* cookie for the attribute name */ +xfblob_cookie attrname_cookie; + +/* cookie for the attribute value */ +xfblob_cookie attrvalue_cookie; };/* Global names storage file. */
@@ -375,6 +398,78 @@ add_parent_ptr( (unsigned long long)ag_pptr.name_cookie); } +/* Remove garbage extended attributes that have ATTR_PARENT set. */ +static void +remove_garbage_xattrs ( +struct xfs_inode *ip, +struct file_scan *fscan) +{ +struct xfs_slab_cursor *cur; +struct garbage_xattr *ga; +void *buf =NULL ; +size_t bufsize =0 ; +int error; + + error = -init_slab_cursor (fscan->garbage_xattr_recs,NULL , &cur); +if (error) +do_error (_ ("init garbage xattr cursor failed: %s\n" ), +strerror (error)); + +while ((ga =pop_slab_cursor (cur)) !=NULL ) { +struct xfs_da_args args = { + .dp = ip, + .attr_filter = ga->attr_filter, + .namelen = ga->attrnamelen, + .valuelen = ga->attrvaluelen, + .op_flags =XFS_DA_OP_REMOVE |XFS_DA_OP_NVLOOKUP , + }; +size_t desired = ga->attrnamelen + ga->attrvaluelen; + +if (desired > bufsize) { +free (buf); + buf =malloc (desired); +if (!buf) +do_error ( +_ ("allocating %zu bytes to remove ino %llu garbage xattr failed: %s\n" ), + desired, + (unsigned long long )ip->i_ino, +strerror (errno)); + bufsize = desired; + } + + args.name = buf; + args.value = buf + ga->attrnamelen; + + error = -xfblob_load (fscan->garbage_xattr_names, + ga->attrname_cookie, buf, ga->attrnamelen); +if (error) +do_error ( +_ ("loading garbage xattr name failed: %s\n" ), +strerror (error)); + + error = -xfblob_load (fscan->garbage_xattr_names, + ga->attrvalue_cookie, args.value, + ga->attrvaluelen); +if (error) +do_error ( +_ ("loading garbage xattr value failed: %s\n" ), +strerror (error)); + + error = -libxfs_attr_set (&args); +if (error) +do_error ( +_ ("removing ino %llu garbage xattr failed: %s\n" ), + (unsigned long long )ip->i_ino, +strerror (error)); + } + +free (buf); +free_slab_cursor (&cur); +free_slab (&fscan->garbage_xattr_recs); +xfblob_destroy (fscan->garbage_xattr_names); + fscan->garbage_xattr_names =NULL ; +} +/* Schedule this ATTR_PARENT extended attribute for deletion. */ static void record_garbage_xattr(
@@ -386,6 +481,14 @@ record_garbage_xattr(const void *value, unsigned int valuelen) { + struct garbage_xattr garbage_xattr = { + .attr_filter = attr_filter, + .attrnamelen = namelen, + .attrvaluelen = valuelen, + }; +struct xfs_mount *mp = ip->i_mount; +int error; +if (no_modify) {if (!fscan->have_garbage) do_warn(
@@ -396,13 +499,45 @@ record_garbage_xattr( }if (fscan->have_garbage) -return ; + goto stuffit; fscan->have_garbage = true;do_warn (_ ("deleting garbage parent pointer extended attributes in ino %llu\n" ), (unsigned long long )ip->i_ino); -/* XXX do the work */ + + error = -init_slab (&fscan->garbage_xattr_recs, +sizeof (struct garbage_xattr )); +if (error) +do_error (_ ("init garbage xattr recs failed: %s\n" ), +strerror (error)); + + error = -xfblob_create (mp,"garbage xattr names" , + &fscan->garbage_xattr_names); +if (error) +do_error ("init garbage xattr names failed: %s\n" , +strerror (error)); + +stuffit: + error = -xfblob_store (fscan->garbage_xattr_names, + &garbage_xattr.attrname_cookie, name, namelen); +if (error) +do_error (_ ("storing ino %llu garbage xattr failed: %s\n" ), + (unsigned long long )ip->i_ino, +strerror (error)); + + error = -xfblob_store (fscan->garbage_xattr_names, + &garbage_xattr.attrvalue_cookie, value, valuelen); +if (error) +do_error (_ ("storing ino %llu garbage xattr failed: %s\n" ), + (unsigned long long )ip->i_ino, +strerror (error)); + + error = -slab_add (fscan->garbage_xattr_recs, &garbage_xattr); +if (error) +do_error (_ ("storing ino %llu garbage xattr rec failed: %s\n" ), + (unsigned long long )ip->i_ino, +strerror (error)); }/*
@@ -922,6 +1057,9 @@ check_file_parent_ptrs( goto out_free; } +if (!no_modify && fscan->have_garbage) +remove_garbage_xattrs (ip, fscan); +crosscheck_file_parent_ptrs (ip, fscan); out_free:
PATCH v11 6/7 xfs_repair: update ondisk parent pointer records
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Update the ondisk parent pointer records as necessary.
--- libxfs/libxfs_api_defs.h | 3 ++ repair/pptr.c | 85 ++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 85 insertions(+), 3 deletions(-)
libxfs/libxfs_api_defs.h +3 -0
@@ -147,9 +147,12 @@#define xfs_parent_add libxfs_parent_add#define xfs_parent_finish libxfs_parent_finish#define xfs_parent_irec_from_disk libxfs_parent_irec_from_disk +#define xfs_parent_irec_hashname libxfs_parent_irec_hashname +#define xfs_parent_set libxfs_parent_set#define xfs_parent_start libxfs_parent_start#define xfs_parent_namecheck libxfs_parent_namecheck#define xfs_parent_valuecheck libxfs_parent_valuecheck +#define xfs_parent_unset libxfs_parent_unset#define xfs_perag_get libxfs_perag_get#define xfs_perag_put libxfs_perag_put#define xfs_prealloc_blocks libxfs_prealloc_blocks
repair/pptr.c +82 -3
109 lines not on this page · raw message
PATCH v11 7/7 xfs_repair: wipe ondisk parent pointers when there are none
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Erase all the parent pointers when there aren't any found by the directory entry scan.
--- repair/pptr.c | 41 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 38 insertions(+), 3 deletions(-)
repair/pptr.c +38 -3
@@ -715,8 +715,13 @@ remove_file_pptr(/* Remove all pptrs from @ip. */ static void clear_all_pptrs ( -struct xfs_inode *ip) +struct xfs_inode *ip, +struct file_scan *fscan) { +struct xfs_slab_cursor *cur; +struct file_pptr *file_pptr; +int error; +if (no_modify) {do_warn (_ ("would delete unlinked ino %llu parent pointers\n" ), (unsigned long long )ip->i_ino);
@@ -725,7 +730,37 @@ clear_all_pptrs(do_warn (_ ("deleting unlinked ino %llu parent pointers\n" ), (unsigned long long )ip->i_ino); -/* XXX actually do the work */ + + error = -init_slab_cursor (fscan->file_pptr_recs,NULL , &cur); +if (error) +do_error (_ ("init ino %llu pptr cursor failed: %s\n" ), + (unsigned long long )ip->i_ino, +strerror (error)); + +while ((file_pptr =pop_slab_cursor (cur)) !=NULL ) { +unsigned char name[MAXNAMELEN ]; + + error =load_file_pptr_name (fscan, file_pptr, name); +if (error) +do_error ( +_ ("loading incorrect name for ino %llu parent pointer (ino %llu gen 0x%x namecookie 0x%llx) failed: %s\n" ), + (unsigned long long )ip->i_ino, + (unsigned long long )file_pptr->parent_ino, + file_pptr->parent_gen, + (unsigned long long )file_pptr->name_cookie, +strerror (error)); + + error =remove_file_pptr (ip, file_pptr, name); +if (error) +do_error ( +_ ("wiping ino %llu pptr (ino %llu gen 0x%x) failed: %s\n" ), + (unsigned long long )ip->i_ino, + (unsigned long long )file_pptr->parent_ino, + file_pptr->parent_gen, +strerror (error)); + } + +free_slab_cursor (&cur); }/* Add @ag_pptr to @ip. */
@@ -994,7 +1029,7 @@ crosscheck_file_parent_ptrs(* file. */ if (fscan->nr_file_pptrs >0 ) -clear_all_pptrs (ip); +clear_all_pptrs (ip, fscan);return ; }
v11 cover fstests: adjust tests for xfs parent pointers
In reply to Darrick J. WongHi all,
These are the test adjustments that are required for parent pointers. There's also a few new tests to ensure that the GETPARENTS ioctl (and hence the ondisk parent pointers) work the way they're supposed to.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
---
common/parent | 209 +++++++
common/populate | 38 +
common/rc | 7
common/xfs | 12
doc/group-names.txt | 1
src/popdir.pl | 11
tests/generic/050 | 10
tests/generic/050.cfg | 1
tests/generic/050.out.xfsquotaparent | 23 +
tests/xfs/018 | 7
tests/xfs/021 | 15 -
tests/xfs/021.cfg | 1
tests/xfs/021.out.default | 0
tests/xfs/021.out.parent | 62 ++
tests/xfs/122.out | 3
tests/xfs/191 | 7
tests/xfs/206 | 3
tests/xfs/288 | 7
tests/xfs/306 | 9
tests/xfs/851 | 116 ++++
tests/xfs/851.out | 69 ++
tests/xfs/852 | 69 ++
tests/xfs/852.out | 1002 ++++++++++++++++++++++++++++++++++
tests/xfs/853 | 85 +++
tests/xfs/853.out | 14
25 files changed, 1770 insertions(+), 11 deletions(-)
create mode 100644 common/parent
create mode 100644 tests/generic/050.out.xfsquotaparent
create mode 100644 tests/xfs/021.cfg
rename tests/xfs/{021.out => 021.out.default} (100%)
create mode 100644 tests/xfs/021.out.parent
create mode 100755 tests/xfs/851
create mode 100644 tests/xfs/851.out
create mode 100755 tests/xfs/852
create mode 100644 tests/xfs/852.out
create mode 100755 tests/xfs/853
create mode 100644 tests/xfs/853.outPATCH v11 1/11 xfs/206: filter out the parent= status from mkfs
From: Darrick J. Wong <[email protected]>
Filter out the parent pointer bits from the mkfs output so that we don't cause a regression in this test.
--- tests/xfs/206 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/xfs/206 +2 -1
@@ -64,7 +64,8 @@ mkfs_filter() -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \ -e "s/, lazy-count=[0-9]//" \ -e "/.*crc=/d" \ - -e "/^Default configuration/d" + -e "/^Default configuration/d" \ + -e '/parent=/d' } # mkfs slightly smaller than that, small log for speed.
PATCH v11 2/11 xfs/122: update for parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Update test for parent pointers.
--- tests/xfs/122.out | 3 +++ 1 file changed, 3 insertions(+)
tests/xfs/122.out +3 -0
@@ -98,6 +98,8 @@ sizeof(struct xfs_fsop_ag_resblks) = 64 sizeof(struct xfs_fsop_geom) = 256 sizeof(struct xfs_fsop_geom_v1) = 112 sizeof(struct xfs_fsop_geom_v4) = 112 +sizeof(struct xfs_getparents) = 96 +sizeof(struct xfs_getparents_rec) = 24 sizeof(struct xfs_icreate_log) = 28 sizeof(struct xfs_inode_log_format) = 56 sizeof(struct xfs_inode_log_format_32) = 52
@@ -107,6 +109,7 @@ sizeof(struct xfs_legacy_timestamp) = 8 sizeof(struct xfs_log_dinode) = 176 sizeof(struct xfs_log_legacy_timestamp) = 8 sizeof(struct xfs_map_extent) = 32 +sizeof(struct xfs_parent_name_rec) = 16 sizeof(struct xfs_phys_extent) = 16 sizeof(struct xfs_refcount_key) = 4 sizeof(struct xfs_refcount_rec) = 12
PATCH v11 3/11 populate: create hardlinks for parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create some hardlinked files so that we can exercise parent pointers.
--- common/populate | 38 ++++++++++++++++++++++++++++++++++++++ src/popdir.pl | 11 +++++++++++ 2 files changed, 49 insertions(+)
common/populate +38 -0
@@ -450,6 +450,44 @@ _scratch_xfs_populate() { cp --reflink=always "${SCRATCH_MNT}/REFCOUNTBT" "${SCRATCH_MNT}/REFCOUNTBT2" fi + # Parent pointers + is_pptr="$(_xfs_has_feature "$SCRATCH_MNT" parent -v)" + if [ $is_pptr -gt 0 ]; then + echo "+ parent pointers" + + # Create a couple of parent pointers + __populate_create_dir "${SCRATCH_MNT}/PPTRS" 1 '' --hardlink --format "two_%d" + + # Create one xattr leaf block of parent pointers + nr="$((blksz * 2 / 16))" + __populate_create_dir "${SCRATCH_MNT}/PPTRS" ${nr} '' --hardlink --format "many%04d" + + # Create multiple xattr leaf blocks of large parent pointers + nr="$((blksz * 16 / 16))" + __populate_create_dir "${SCRATCH_MNT}/PPTRS" ${nr} '' --hardlink --format "y%0254d" + + # Create multiple paths to a file + local moof="${SCRATCH_MNT}/PPTRS/moofile" + touch "${moof}" + for ((i = 0; i < 4; i++)); do + mkdir -p "${SCRATCH_MNT}/PPTRS/SUB${i}" + ln "${moof}" "${SCRATCH_MNT}/PPTRS/SUB${i}/moofile" + done + + # Create parent pointers of various lengths + touch "${SCRATCH_MNT}/PPTRS/vlength" + local len_len + local tst + local fname + ln "${SCRATCH_MNT}/PPTRS/vlength" "${SCRATCH_MNT}/PPTRS/b" + for len in 32 64 96 128 160 192 224 250 255; do + len_len="${#len}" + tst="$(perl -e "print \"b\" x (${len} - (${len_len} + 1))")" + fname="v${tst}${len}" + ln "${SCRATCH_MNT}/PPTRS/vlength" "${SCRATCH_MNT}/PPTRS/${fname}" + done + fi + # Copy some real files (xfs tests, I guess...) echo "+ real files" test $fill -ne 0 && __populate_fill_fs "${SCRATCH_MNT}" 5
src/popdir.pl +11 -0
@@ -17,6 +17,7 @@ GetOptions("start=i" => \$start, "dir=s" => \$dir, "remove!" => \$remove, "help!" => \$help, + "hardlink!" => \$hardlink, "verbose!" => \$verbose);
@@ -35,6 +36,7 @@ Options: --format=str printf formatting string for file name ("%08d") --verbose verbose output --help this help screen + --hardlink hardlink subsequent files to the first one created EOF exit(1) unless defined $help; # otherwise...
@@ -56,12 +58,21 @@ if ($file_pct < 0) { $file_pct = 100; } +if ($hardlink) { + $file_pct = 100; + $link_fname = sprintf($format, $start); +} + for ($i = $start; $i <= $end; $i += $incr) { $fname = sprintf($format, $i); if ($remove) { $verbose && print "rm $fname\n"; unlink($fname) or rmdir($fname) or die("unlink $fname"); + } elsif ($hardlink && $i > $start) { + # hardlink everything after the first file + $verbose && print "ln $link_fname $fname\n"; + link $link_fname, $fname; } elsif (($i % 100) < $file_pct) { # create a file $verbose && print "touch $fname\n";
PATCH v11 4/11 xfs/021: adapt golden output files for parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Parent pointers change the xattr structure dramatically, so fix this test to handle them. For the most part we can get away with filtering out the parent pointer fields (which xfs_db decodes for us), but the namelen/valuelen/attr_filter fields still show through.
---
common/rc | 4 +++
tests/xfs/021 | 15 +++++++++--
tests/xfs/021.cfg | 1 +
tests/xfs/021.out.default | 0
tests/xfs/021.out.parent | 62 +++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 80 insertions(+), 2 deletions(-)
create mode 100644 tests/xfs/021.cfg
rename tests/xfs/{021.out => 021.out.default} (100%)
create mode 100644 tests/xfs/021.out.parentcommon/rc +4 -0
@@ -3307,6 +3307,8 @@ _get_os_name() _link_out_file_named() { + test -n "$seqfull" || _fail "need to set seqfull" + local features=$2 local suffix=$(FEATURES="$features" perl -e ' my %feathash;
@@ -3342,6 +3344,8 @@ _link_out_file() { local features + test -n "$seqfull" || _fail "need to set seqfull" + if [ $# -eq 0 ]; then features="$(_get_os_name),$FSTYP" if [ -n "$MOUNT_OPTIONS" ]; then
tests/xfs/021 +13 -2
@@ -67,6 +67,13 @@ _scratch_mkfs_xfs >/dev/null \ echo "*** mount FS" _scratch_mount +seqfull=$0 +if _xfs_has_feature $SCRATCH_MNT parent; then + _link_out_file "parent" +else + _link_out_file "" +fi + testfile=$SCRATCH_MNT/testfile echo "*** make test file 1"
@@ -108,7 +115,10 @@ _scratch_unmount >>$seqres.full 2>&1 \ echo "*** dump attributes (1)" _scratch_xfs_db -r -c "inode $inum_1" -c "print a.sfattr" | \ - sed -e '/secure = /d' | sed -e '/parent = /d' + perl -ne ' +/\.secure/ && next; +/\.parent/ && next; + print unless /^\d+:\[.*/;' echo "*** dump attributes (2)"
@@ -124,10 +134,11 @@ s/info.hdr/info/; /hdr.info.uuid/ && next; /hdr.info.lsn/ && next; /hdr.info.owner/ && next; +/\.parent/ && next; s/^(hdr.info.magic =) 0x3bee/\1 0xfbee/; s/^(hdr.firstused =) (\d+)/\1 FIRSTUSED/; s/^(hdr.freemap\[0-2] = \[base,size]).*/\1 [FREEMAP..]/; -s/^(entries\[0-2] = \[hashval,nameidx,incomplete,root,local]).*/\1 [ENTRIES..]/; +s/^(entries\[0-[23]] = \[hashval,nameidx,incomplete,root,local]).*/\1 [ENTRIES..]/; print unless /^\d+:\[.*/;' echo "*** done"
tests/xfs/021.cfg new file +1 -0
@@ -0,0 +1 @@ +parent: parent
tests/xfs/021.out.default renamed from tests/xfs/021.out +0 -0
tests/xfs/021.out.parent new file +62 -0
62 lines not on this page · raw message
PATCH v11 5/11 generic/050: adapt for parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Fix this test when quotas and parent pointers are enabled.
--- tests/generic/050 | 10 ++++++++++ tests/generic/050.cfg | 1 + tests/generic/050.out.xfsquotaparent | 23 +++++++++++++++++++++++ 3 files changed, 34 insertions(+) create mode 100644 tests/generic/050.out.xfsquotaparent
tests/generic/050 +10 -0
@@ -36,6 +36,16 @@ elif [ "$FSTYP" = "xfs" ] && echo "$MOUNT_OPTIONS" | grep -q quota ; then # Mounting with quota on XFS requires a writable fs, which means # we expect to fail the ro blockdev test with with EPERM. features="xfsquota" + + if _xfs_has_feature $SCRATCH_DEV parent && \ + ! _xfs_has_feature $SCRATCH_DEV realtime; then + # If we have quotas and parent pointers enabled, the primary + # superblock will be written out with the quota flags set when + # the logged xattrs log_incompat feature is set. Hence the + # norecovery mount won't fail due to quota rejecting the + # mismatch between the mount qflags and the ondisk ones. + features="xfsquotaparent" + fi fi _link_out_file "$features"
tests/generic/050.cfg +1 -0
@@ -1,2 +1,3 @@ nojournal: nojournal xfsquota: xfsquota +xfsquotaparent: xfsquotaparent
tests/generic/050.out.xfsquotaparent new file +23 -0
@@ -0,0 +1,23 @@ +QA output created by 050 +setting device read-only +mounting read-only block device: +mount: SCRATCH_MNT: permission denied +unmounting read-only filesystem +umount: SCRATCH_DEV: not mounted +setting device read-write +mounting read-write block device: +touch files +going down: +unmounting shutdown filesystem: +setting device read-only +mounting filesystem that needs recovery on a read-only device: +mount: device write-protected, mounting read-only +mount: cannot mount device read-only +unmounting read-only filesystem +umount: SCRATCH_DEV: not mounted +mounting filesystem with -o norecovery on a read-only device: +mount: device write-protected, mounting read-only +unmounting read-only filesystem +setting device read-write +mounting filesystem that needs recovery with -o ro: +*** done