xfs: directory parent pointers
Subsystems (2)
Hi everyone,
Christoph and I have been working to get the directory parent pointers patchset into shape for merging in the next kernel cycle. This v13 release contains what I hope are the last ondisk format changes -- we've gone back to parent pointers being xattrs attached to the child, wherein the attr name is the dirent name, and the attr value is a handle to the parent directory.
We've solved the pptr lookup uniqueness problem by forcing all XFS_ATTR_PARENT attr lookups to be done on the name and value; avoided namehash collisions on container farms by adjusting the hash function slightly; and avoided the entire log incompat feature flag mess by defining a permanent incompat feature for parent pointers and using totally separate attr log item opcodes.
With that, I think this is finally ready to go.
Full versions are here:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=vectorized-scrub
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub
--D
PATCH 1/9 xfs: design documentation for online fsck, part 2
Hi all,
This series updates the design documentation for online fsck to reflect the final design of the parent pointers feature as well as the implementation of online fsck for the new metadata.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=online-fsck-design
---
Commits in this patchset:
* docs: update the parent pointers documentation to the final version
* docs: update online directory and parent pointer repair sections
* docs: update offline parent pointer repair strategy
* docs: describe xfs directory tree online fsck
--- .../filesystems/xfs/xfs-online-fsck-design.rst | 354 +++++++++++++++----- 1 file changed, 266 insertions(+), 88 deletions(-)
PATCH 1/4 docs: update the parent pointers documentation to the final version
From: Darrick J. Wong <[email protected]>
Now that we've decided on the ondisk format of parent pointers, update the documentation to reflect that.
--- .../filesystems/xfs/xfs-online-fsck-design.rst | 94 +++++++++++--------- 1 file changed, 53 insertions(+), 41 deletions(-)
Documentation/filesystems/xfs/xfs-online-fsck-design.rst +53 -41
@@ -4465,10 +4465,10 @@ reconstruction of filesystem space metadata. The parent pointer feature, however, makes total directory reconstruction possible. -XFS parent pointers include the dirent name and location of the entry within -the parent directory. +XFS parent pointers contain the information needed to identify the +corresponding directory entry in the parent directory. In other words, child files use extended attributes to store pointers to -parents in the form ``(parent_inum, parent_gen, dirent_pos) → (dirent_name)``. +parents in the form ``(dirent_name) → (parent_inum, parent_gen)``. The directory checking process can be strengthened to ensure that the target of each dirent also contains a parent pointer pointing back to the dirent. Likewise, each parent pointer can be checked by ensuring that the target of
@@ -4476,8 +4476,6 @@ each parent pointer is a directory and that it contains a dirent matching the parent pointer. Both online and offline repair can use this strategy. -**Note**: The ondisk format of parent pointers is not yet finalized. - +--------------------------------------------------------------------------+ | **Historical Sidebar**: | +--------------------------------------------------------------------------+
@@ -4519,8 +4517,58 @@ Both online and offline repair can use this strategy. | Chandan increased the maximum extent counts of both data and attribute | | forks, thereby ensuring that the extended attribute structure can grow | | to handle the maximum hardlink count of any file. | +| | +| For this second effort, the ondisk parent pointer format as originally | +| proposed was ``(parent_inum, parent_gen, dirent_pos) → (dirent_name)``. | +| The format was changed during development to eliminate the requirement | +| of repair tools needing to to ensure that the ``dirent_pos`` field | +| always matched when reconstructing a directory. | +| | +| There were a few other ways to have solved that problem: | +| | +| 1. The field could be designated advisory, since the other three values | +| are sufficient to find the entry in the parent. | +| However, this makes indexed key lookup impossible while repairs are | +| ongoing. | +| | +| 2. We could allow creating directory entries at specified offsets, which | +| solves the referential integrity problem but runs the risk that | +| dirent creation will fail due to conflicts with the free space in the | +| directory. | +| | +| These conflicts could be resolved by appending the directory entry | +| and amending the xattr code to support updating an xattr key and | +| reindexing the dabtree, though this would have to be performed with | +| the parent directory still locked. | +| | +| 3. Same as above, but remove the old parent pointer entry and add a new | +| one atomically. | +| | +| 4. Change the ondisk xattr format to | +| ``(parent_inum, name) → (parent_gen)``, which would provide the attr | +| name uniqueness that we require, without forcing repair code to | +| update the dirent position. | +| Unfortunately, this requires changes to the xattr code to support | +| attr names as long as 263 bytes. | +| | +| 5. Change the ondisk xattr format to ``(parent_inum, hash(name)) → | +| (name, parent_gen)``. | +| If the hash is sufficiently resistant to collisions (e.g. sha256) | +| then this should provide the attr name uniqueness that we require. | +| Names shorter than 247 bytes could be stored directly. | +| | +| 6. Change the ondisk xattr format to ``(dirent_name) → (parent_ino, | +| parent_gen)``. This format doesn't require any of the complicated | +| nested name hashing of the previous suggestions. However, it was | +| discovered that multiple hardlinks to the same inode with the same | +| filename caused performance problems with hashed xattr lookups, so | +| the parent inumber is now xor'd into the hash index. | +| | +| In the end, it was decided that solution #6 was the most compact and the | +| most performant. A new hash function was designed for parent pointers. | +--------------------------------------------------------------------------+ + Case Study: Repairing Directories with Parent Pointers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -4569,42 +4617,6 @@ The proposed patchset is the <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-repair>`_ series. -**Unresolved Question**: How will repair ensure that the ``dirent_pos`` fields -match in the reconstructed directory? - -*Answer*: There are a few ways to solve this problem: - -1. The field could be designated advisory, since the other three values are - sufficient to find the entry in the parent. - However, this makes indexed key lookup impossible while repairs are ongoing. - -2. We could allow creating directory entries at specified offsets, which solves - the referential integrity problem but runs the risk that dirent creation - will fail due to conflicts with the free space in the directory. - - These conflicts could be resolved by appending the directory entry and - amending the xattr code to support updating an xattr key and reindexing the - dabtree, though this would have to be performed with the parent directory - still locked. - -3. Same as above, but remove the old parent pointer entry and add a new one - atomically. - -4. Change the ondisk xattr format to ``(parent_inum, name) → (parent_gen)``, - which would provide the attr name uniqueness that we require, without - forcing repair code to update the dirent position. - Unfortunately, this requires changes to the xattr code to support attr - names as long as 263 bytes. - -5. Change the ondisk xattr format to ``(parent_inum, hash(name)) → - (name, parent_gen)``. - If the hash is sufficiently resistant to collisions (e.g. sha256) then - this should provide the attr name uniqueness that we require. - Names shorter than 247 bytes could be stored directly. - -Discussion is ongoing under the `parent pointers patch deluge -<https://www.spinics.net/lists/linux-xfs/msg69397.html>`_. - Case Study: Repairing Parent Pointers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Re: [PATCH 1/4] docs: update the parent pointers documentation to the final version
Looks good:
PATCH 2/4 docs: update online directory and parent pointer repair sections
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Update the case studies of online directory and parent pointer reconstruction to reflect what they actually do in the final version.
--- .../filesystems/xfs/xfs-online-fsck-design.rst | 55 +++++++++++--------- 1 file changed, 29 insertions(+), 26 deletions(-)
Documentation/filesystems/xfs/xfs-online-fsck-design.rst +29 -26
@@ -4576,8 +4576,9 @@ Directory rebuilding uses a :ref:`coordinated inode scan <iscan>` and a :ref:`directory entry live update hook <liveupdate>` as follows: 1. Set up a temporary directory for generating the new directory structure, - an xfblob for storing entry names, and an xfarray for stashing directory - updates. + an xfblob for storing entry names, and an xfarray for stashing the fixed + size fields involved in a directory update: ``(child inumber, add vs. + remove, name cookie, ftype)``. 2. Set up an inode scanner and hook into the directory entry code to receive updates on directory operations.
@@ -4586,35 +4587,34 @@ a :ref:`directory entry live update hook <liveupdate>` as follows: pointer references the directory of interest. If so: - a. Stash an addname entry for this dirent in the xfarray for later. + a. Stash the parent pointer name and an addname entry for this dirent in the + xfblob and xfarray, respectively. - b. When finished scanning that file, flush the stashed updates to the - temporary directory. + b. When finished scanning that file or the kernel memory consumption exceeds + a threshold, flush the stashed updates to the temporary directory. 4. For each live directory update received via the hook, decide if the child has already been scanned. If so: - a. Stash an addname or removename entry for this dirent update in the - xfarray for later. + a. Stash the parent pointer name an addname or removename entry for this + dirent update in the xfblob and xfarray for later. We cannot write directly to the temporary directory because hook functions are not allowed to modify filesystem metadata. Instead, we stash updates in the xfarray and rely on the scanner thread to apply the stashed updates to the temporary directory. -5. When the scan is complete, atomically exchange the contents of the temporary +5. When the scan is complete, replay any stashed entries in the xfarray. + +6. When the scan is complete, atomically exchange the contents of the temporary directory and the directory being repaired. The temporary directory now contains the damaged directory structure. -6. Reap the temporary directory. - -7. Update the dirent position field of parent pointers as necessary. - This may require the queuing of a substantial number of xattr log intent - items. +7. Reap the temporary directory. The proposed patchset is the `parent pointers directory repair -<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-repair>`_ +<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-fsck>`_ series. Case Study: Repairing Parent Pointers
@@ -4624,8 +4624,9 @@ Online reconstruction of a file's parent pointer information works similarly to directory reconstruction: 1. Set up a temporary file for generating a new extended attribute structure, - an `xfblob<xfblob>` for storing parent pointer names, and an xfarray for - stashing parent pointer updates. + an xfblob for storing parent pointer names, and an xfarray for stashing the + fixed size fields involved in a parent pointer update: ``(parent inumber, + parent generation, add vs. remove, name cookie)``. 2. Set up an inode scanner and hook into the directory entry code to receive updates on directory operations.
@@ -4634,34 +4635,36 @@ directory reconstruction: dirent references the file of interest. If so: - a. Stash an addpptr entry for this parent pointer in the xfblob and xfarray - for later. + a. Stash the dirent name and an addpptr entry for this parent pointer in the + xfblob and xfarray, respectively. - b. When finished scanning the directory, flush the stashed updates to the - temporary directory. + b. When finished scanning the directory or the kernel memory consumption + exceeds a threshold, flush the stashed updates to the temporary file. 4. For each live directory update received via the hook, decide if the parent has already been scanned. If so: - a. Stash an addpptr or removepptr entry for this dirent update in the - xfarray for later. + a. Stash the dirent name and an addpptr or removepptr entry for this dirent + update in the xfblob and xfarray for later. We cannot write parent pointers directly to the temporary file because hook functions are not allowed to modify filesystem metadata. Instead, we stash updates in the xfarray and rely on the scanner thread to apply the stashed parent pointer updates to the temporary file. -5. Copy all non-parent pointer extended attributes to the temporary file. +5. When the scan is complete, replay any stashed entries in the xfarray. -6. When the scan is complete, atomically exchange the mappings of the attribute +6. Copy all non-parent pointer extended attributes to the temporary file. + +7. When the scan is complete, atomically exchange the mappings of the attribute forks of the temporary file and the file being repaired. The temporary file now contains the damaged extended attribute structure. -7. Reap the temporary file. +8. Reap the temporary file. The proposed patchset is the `parent pointers repair -<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-repair>`_ +<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-fsck>`_ series. Digression: Offline Checking of Parent Pointers
Re: [PATCH 2/4] docs: update online directory and parent pointer repair sections
Looks good:
PATCH 3/4 docs: update offline parent pointer repair strategy
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now update how xfs_repair checks and repairs parent pointer info.
--- .../filesystems/xfs/xfs-online-fsck-design.rst | 81 +++++++++++++++----- 1 file changed, 60 insertions(+), 21 deletions(-)
Documentation/filesystems/xfs/xfs-online-fsck-design.rst +60 -21
@@ -4675,26 +4675,56 @@ files are erased long before directory tree connectivity checks are performed. Parent pointer checks are therefore a second pass to be added to the existing connectivity checks: -1. After the set of surviving files has been established (i.e. phase 6), +1. After the set of surviving files has been established (phase 6), walk the surviving directories of each AG in the filesystem. This is already performed as part of the connectivity checks. -2. For each directory entry found, record the name in an xfblob, and store - ``(child_ag_inum, parent_inum, parent_gen, dirent_pos)`` tuples in a - per-AG in-memory slab. +2. For each directory entry found, + + a. If the name has already been stored in the xfblob, then use that cookie + and skip the next step. + + b. Otherwise, record the name in an xfblob, and remember the xfblob cookie. + Unique mappings are critical for + + 1. Deduplicating names to reduce memory usage, and + + 2. Creating a stable sort key for the parent pointer indexes so that the + parent pointer validation described below will work. + + c. Store ``(child_ag_inum, parent_inum, parent_gen, name_hash, name_len, + name_cookie)`` tuples in a per-AG in-memory slab. The ``name_hash`` + referenced in this section is the regular directory entry name hash, not + the specialized one used for parent pointer xattrs. 3. For each AG in the filesystem, - a. Sort the per-AG tuples in order of child_ag_inum, parent_inum, and - dirent_pos. + a. Sort the per-AG tuple set in order of ``child_ag_inum``, ``parent_inum``, + ``name_hash``, and ``name_cookie``. + Having a single ``name_cookie`` for each ``name`` is critical for + handling the uncommon case of a directory containing multiple hardlinks + to the same file where all the names hash to the same value. b. For each inode in the AG, 1. Scan the inode for parent pointers. - Record the names in a per-file xfblob, and store ``(parent_inum, - parent_gen, dirent_pos)`` tuples in a per-file slab. + For each parent pointer found, - 2. Sort the per-file tuples in order of parent_inum, and dirent_pos. + a. Validate the ondisk parent pointer. + If validation fails, move on to the next parent pointer in the + file. + + b. If the name has already been stored in the xfblob, then use that + cookie and skip the next step. + + c. Record the name in a per-file xfblob, and remember the xfblob + cookie. + + d. Store ``(parent_inum, parent_gen, name_hash, name_len, + name_cookie)`` tuples in a per-file slab. + + 2. Sort the per-file tuples in order of ``parent_inum``, ``name_hash``, + and ``name_cookie``. 3. Position one slab cursor at the start of the inode's records in the per-AG tuple slab.
@@ -4703,28 +4733,37 @@ connectivity checks: 4. Position a second slab cursor at the start of the per-file tuple slab. - 5. Iterate the two cursors in lockstep, comparing the parent_ino and - dirent_pos fields of the records under each cursor. + 5. Iterate the two cursors in lockstep, comparing the ``parent_ino``, + ``name_hash``, and ``name_cookie`` fields of the records under each + cursor: - a. Tuples in the per-AG list but not the per-file list are missing and - need to be written to the inode. + a. If the per-AG cursor is at a lower point in the keyspace than the + per-file cursor, then the per-AG cursor points to a missing parent + pointer. + Add the parent pointer to the inode and advance the per-AG + cursor. - b. Tuples in the per-file list but not the per-AG list are dangling - and need to be removed from the inode. + b. If the per-file cursor is at a lower point in the keyspace than + the per-AG cursor, then the per-file cursor points to a dangling + parent pointer. + Remove the parent pointer from the inode and advance the per-file + cursor. - c. For tuples in both lists, update the parent_gen and name components - of the parent pointer if necessary. + c. Otherwise, both cursors point at the same parent pointer. + Update the parent_gen component if necessary. + Advance both cursors. 4. Move on to examining link counts, as we do today. The proposed patchset is the `offline parent pointers repair -<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-repair>`_ +<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-fsck>`_ series. -Rebuilding directories from parent pointers in offline repair is very -challenging because it currently uses a single-pass scan of the filesystem -during phase 3 to decide which files are corrupt enough to be zapped. +Rebuilding directories from parent pointers in offline repair would be very +challenging because xfs_repair currently uses two single-pass scans of the +filesystem during phases 3 and 4 to decide which files are corrupt enough to be +zapped. This scan would have to be converted into a multi-pass scan: 1. The first pass of the scan zaps corrupt inodes, forks, and attributes
Re: [PATCH 3/4] docs: update offline parent pointer repair strategy
Looks good:
PATCH 4/4 docs: describe xfs directory tree online fsck
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
I've added a scrubber that checks the directory tree structure and fixes them; describe this in the design documentation.
--- .../filesystems/xfs/xfs-online-fsck-design.rst | 124 ++++++++++++++++++++ 1 file changed, 124 insertions(+)
Documentation/filesystems/xfs/xfs-online-fsck-design.rst +124 -0
@@ -4785,6 +4785,130 @@ This scan would have to be converted into a multi-pass scan: This code has not yet been constructed. +.. _dirtree: + +Case Study: Directory Tree Structure +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +As mentioned earlier, the filesystem directory tree is supposed to be a +directed acylic graph structure. +However, each node in this graph is a separate ``xfs_inode`` object with its +own locks, which makes validating the tree qualities difficult. +Fortunately, non-directories are allowed to have multiple parents and cannot +have children, so only directories need to be scanned. +Directories typically constitute 5-10% of the files in a filesystem, which +reduces the amount of work dramatically. + +If the directory tree could be frozen, it would be easy to discover cycles and +disconnected regions by running a depth (or breadth) first search downwards +from the root directory and marking a bitmap for each directory found. +At any point in the walk, trying to set an already set bit means there is a +cycle. +After the scan completes, XORing the marked inode bitmap with the inode +allocation bitmap reveals disconnected inodes. +However, one of online repair's design goals is to avoid locking the entire +filesystem unless it's absolutely necessary. +Directory tree updates can move subtrees across the scanner wavefront on a live +filesystem, so the bitmap algorithm cannot be applied. + +Directory parent pointers enable an incremental approach to validation of the +tree structure. +Instead of using one thread to scan the entire filesystem, multiple threads can +walk from individual subdirectories upwards towards the root. +For this to work, all directory entries and parent pointers must be internally +consistent, each directory entry must have a parent pointer, and the link +counts of all directories must be correct. +Each scanner thread must be able to take the IOLOCK of an alleged parent +directory while holding the IOLOCK of the child directory to prevent either +directory from being moved within the tree. +This is not possible since the VFS does not take the IOLOCK of a child +subdirectory when moving that subdirectory, so instead the scanner stabilizes +the parent -> child relationship by taking the ILOCKs and installing a dirent +update hook to detect changes. + +The scanning process uses a dirent hook to detect changes to the directories +mentioned in the scan data. +The scan works as follows: + +1. For each subdirectory in the filesystem, + + a. For each parent pointer of that subdirectory, + + 1. Create a path object for that parent pointer, and mark the + subdirectory inode number in the path object's bitmap. + + 2. Record the parent pointer name and inode number in a path structure. + + 3. If the alleged parent is the subdirectory being scrubbed, the path is + a cycle. + Mark the path for deletion and repeat step 1a with the next + subdirectory parent pointer. + + 4. Try to mark the alleged parent inode number in a bitmap in the path + object. + If the bit is already set, then there is a cycle in the directory + tree. + Mark the path as a cycle and repeat step 1a with the next subdirectory + parent pointer. + + 5. Load the alleged parent. + If the alleged parent is not a linked directory, abort the scan + because the parent pointer information is inconsistent. + + 6. For each parent pointer of this alleged ancestor directory, + + a. Record the parent pointer name and inode number in the path object + if no parent has been set for that level. + + b. If an ancestor has more than one parent, mark the path as corrupt. + Repeat step 1a with the next subdirectory parent pointer. + + c. Repeat steps 1a3-1a6 for the ancestor identified in step 1a6a. + This repeats until the directory tree root is reached or no parents + are found. + + 7. If the walk terminates at the root directory, mark the path as ok. + + 8. If the walk terminates without reaching the root, mark the path as + disconnected. + +2. If the directory entry update hook triggers, check all paths already found + by the scan. + If the entry matches part of a path, mark that path and the scan stale. + When the scanner thread sees that the scan has been marked stale, it deletes + all scan data and starts over. + +Repairing the directory tree works as follows: + +1. Walk each path of the target subdirectory. + + a. Corrupt paths and cycle paths are counted as suspect. + + b. Paths already marked for deletion are counted as bad. + + c. Paths that reached the root are counted as good. + +2. If the subdirectory is either the root directory or has zero link count, + delete all incoming directory entries in the immediate parents. + Repairs are complete. + +3. If the subdirectory has exactly one path, set the dotdot entry to the + parent and exit. + +4. If the subdirectory has at least one good path, delete all the other + incoming directory entries in the immediate parents. + +5. If the subdirectory has no good paths and more than one suspect path, delete + all the other incoming directory entries in the immediate parents. + +6. If the subdirectory has zero paths, attach it to the lost and found. + +The proposed patches are in the +`directory tree repair +<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-directory-tree>`_ +series. + + .. _orphanage: The Orphanage
Re: [PATCH 4/4] docs: describe xfs directory tree online fsck
Looks good:
PATCH 2/9 xfs: retain ILOCK during directory updates
In reply to Darrick J. WongHi all,
This series changes the directory update code to retain the ILOCK on all files involved in a rename until the end of the operation. The upcoming parent pointers patchset applies parent pointers in a separate chained update from the actual directory update, which is why it is now necessary to keep the ILOCK instead of dropping it after the first transaction in the chain.
As a side effect, we no longer need to hold the IOLOCK during an rmapbt scan of inodes to serialize the scan with ongoing directory updates.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=retain-ilock-during-dir-ops
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=retain-ilock-during-dir-ops
---
Commits in this patchset:
* xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
* xfs: Increase XFS_QM_TRANS_MAXDQS to 5
* xfs: Hold inode locks in xfs_ialloc
* xfs: Hold inode locks in xfs_trans_alloc_dir
* xfs: Hold inode locks in xfs_rename
* xfs: don't pick up IOLOCK during rmapbt repair scan
* xfs: unlock new repair tempfiles after creation
--- fs/xfs/libxfs/xfs_defer.c | 6 ++- fs/xfs/libxfs/xfs_defer.h | 8 +++- fs/xfs/scrub/rmap_repair.c | 16 ------- fs/xfs/scrub/tempfile.c | 2 + fs/xfs/xfs_dquot.c | 41 ++++++++++++++++++ fs/xfs/xfs_dquot.h | 1 fs/xfs/xfs_inode.c | 98 ++++++++++++++++++++++++++++++++------------ fs/xfs/xfs_inode.h | 2 + fs/xfs/xfs_qm.c | 4 +- fs/xfs/xfs_qm.h | 2 - fs/xfs/xfs_symlink.c | 6 ++- fs/xfs/xfs_trans.c | 9 +++- fs/xfs/xfs_trans_dquot.c | 15 ++++--- 13 files changed, 156 insertions(+), 54 deletions(-)
PATCH 1/7 xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
From: Allison Henderson <[email protected]>
Renames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.
--- fs/xfs/libxfs/xfs_defer.c | 6 +++++- fs/xfs/libxfs/xfs_defer.h | 8 +++++++- fs/xfs/xfs_inode.c | 27 ++++++++++++++++++--------- fs/xfs/xfs_inode.h | 2 ++ 4 files changed, 32 insertions(+), 11 deletions(-)
fs/xfs/libxfs/xfs_defer.c +5 -1
@@ -1092,7 +1092,11 @@ xfs_defer_ops_continue(ASSERT (!(tp->t_flags &XFS_TRANS_DIRTY ));/* Lock the captured resources to the new transaction. */ -if (dfc->dfc_held.dr_inos ==2 ) +if (dfc->dfc_held.dr_inos >2 ) { +xfs_sort_inodes (dfc->dfc_held.dr_ip, dfc->dfc_held.dr_inos); +xfs_lock_inodes (dfc->dfc_held.dr_ip, dfc->dfc_held.dr_inos, +XFS_ILOCK_EXCL ); + }else if (dfc->dfc_held.dr_inos ==2 )xfs_lock_two_inodes (dfc->dfc_held.dr_ip[0 ],XFS_ILOCK_EXCL , dfc->dfc_held.dr_ip[1 ],XFS_ILOCK_EXCL );else if (dfc->dfc_held.dr_inos ==1 )
fs/xfs/libxfs/xfs_defer.h +7 -1
@@ -77,7 +77,13 @@ extern const struct xfs_defer_op_type xfs_exchmaps_defer_type;/* * Deferred operation item relogging limits. */ -#define XFS_DEFER_OPS_NR_INODES 2/* join up to two inodes */ + +/* +* Rename w/ parent pointers can require up to 5 inodes with deferred ops to +* be joined to the transaction: src_dp, target_dp, src_ip, target_ip, and wip. +* These inodes are locked in sorted order by their inode numbers +*/ +#define XFS_DEFER_OPS_NR_INODES 5#define XFS_DEFER_OPS_NR_BUFS 2/* join up to two buffers */ /* Resources that must be held across a transaction roll. */
fs/xfs/xfs_inode.c +18 -9
@@ -418,7 +418,7 @@ xfs_lock_inumorder(* lock more than one at a time, lockdep will report false positives saying we * have violated locking orders. */ -static void +void xfs_lock_inodes( struct xfs_inode **ips, int inodes,
@@ -2802,7 +2802,7 @@ xfs_sort_for_rename(struct xfs_inode **i_tab,/* out: sorted array of inodes */ int *num_inodes) /* in/out: inodes in array */ { - int i, j; + int i;ASSERT (*num_inodes == __XFS_SORT_INODES);memset (i_tab,0 , *num_inodes *sizeof (struct xfs_inode *));
@@ -2824,17 +2824,26 @@ xfs_sort_for_rename( i_tab[i++] = wip; *num_inodes = i; +xfs_sort_inodes (i_tab, *num_inodes); +} + +void +xfs_sort_inodes ( +struct xfs_inode **i_tab, +unsigned int num_inodes) +{ +int i, j; + +ASSERT (num_inodes <= __XFS_SORT_INODES); +/* * Sort the elements via bubble sort. (Remember, there are at * most 5 elements to sort, so this is adequate.) */ -for (i =0 ; i < *num_inodes; i++) { -for (j =1 ; j < *num_inodes; j++) { -if (i_tab[j]->i_ino < i_tab[j-1 ]->i_ino) { -struct xfs_inode *temp = i_tab[j]; - i_tab[j] = i_tab[j-1 ]; - i_tab[j-1 ] = temp; - } +for (i =0 ; i < num_inodes; i++) { +for (j =1 ; j < num_inodes; j++) { +if (i_tab[j]->i_ino < i_tab[j-1 ]->i_ino) +swap (i_tab[j], i_tab[j -1 ]); } } }
fs/xfs/xfs_inode.h +2 -0
@@ -627,6 +627,8 @@ int xfs_ilock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2);void xfs_iunlock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_iunlock2_remapping (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_bumplink (struct xfs_trans *tp,struct xfs_inode *ip); +void xfs_lock_inodes (struct xfs_inode **ips,int inodes,uint lock_mode); +void xfs_sort_inodes (struct xfs_inode **i_tab,unsigned int num_inodes);static inline bool xfs_inode_unlinked_incomplete(
Re: [PATCH 1/7] xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
Looks good:
PATCH 2/7 xfs: Increase XFS_QM_TRANS_MAXDQS to 5
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
With parent pointers enabled, a rename operation can update up to 5
inodes: src_dp, target_dp, src_ip, target_ip and wip. This causes their dquots to a be attached to the transaction chain, so we need to increase XFS_QM_TRANS_MAXDQS. This patch also add a helper function xfs_dqlockn to lock an arbitrary number of dquots.
--- fs/xfs/xfs_dquot.c | 41 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_dquot.h | 1 + fs/xfs/xfs_qm.h | 2 +- fs/xfs/xfs_trans_dquot.c | 15 ++++++++++----- 4 files changed, 53 insertions(+), 6 deletions(-)
fs/xfs/xfs_dquot.c +41 -0
@@ -1371,6 +1371,47 @@ xfs_dqlock2( } } +static int +xfs_dqtrx_cmp ( +const void *a, +const void *b) +{ +const struct xfs_dqtrx *qa = a; +const struct xfs_dqtrx *qb = b; + +if (qa->qt_dquot->q_id > qb->qt_dquot->q_id) +return 1 ; +if (qa->qt_dquot->q_id < qb->qt_dquot->q_id) +return -1 ; +return 0 ; +} + +void +xfs_dqlockn ( +struct xfs_dqtrx *q) +{ +unsigned int i; + +BUILD_BUG_ON (XFS_QM_TRANS_MAXDQS >MAX_LOCKDEP_SUBCLASSES ); + +/* Sort in order of dquot id, do not allow duplicates */ +for (i =0 ; i <XFS_QM_TRANS_MAXDQS && q[i].qt_dquot !=NULL ; i++) { +unsigned int j; + +for (j =0 ; j < i; j++) +ASSERT (q[i].qt_dquot != q[j].qt_dquot); + } +if (i ==0 ) +return ; + +sort (q, i,sizeof (struct xfs_dqtrx ), xfs_dqtrx_cmp,NULL ); + +mutex_lock (&q[0 ].qt_dquot->q_qlock); +for (i =1 ; i <XFS_QM_TRANS_MAXDQS && q[i].qt_dquot !=NULL ; i++) +mutex_lock_nested (&q[i].qt_dquot->q_qlock, +XFS_QLOCK_NESTED + i -1 ); +} +int __init xfs_qm_init (void ) {
fs/xfs/xfs_dquot.h +1 -0
@@ -223,6 +223,7 @@ int xfs_qm_dqget_uncached(struct xfs_mount *mp,void xfs_qm_dqput (struct xfs_dquot *dqp);void xfs_dqlock2 (struct xfs_dquot *,struct xfs_dquot *); +void xfs_dqlockn (struct xfs_dqtrx *q);void xfs_dquot_set_prealloc_limits (struct xfs_dquot *);
fs/xfs/xfs_qm.h +1 -1
@@ -136,7 +136,7 @@ enum {XFS_QM_TRANS_PRJ , XFS_QM_TRANS_DQTYPES }; -#define XFS_QM_TRANS_MAXDQS 2 +#define XFS_QM_TRANS_MAXDQS 5struct xfs_dquot_acct {struct xfs_dqtrx dqs[XFS_QM_TRANS_DQTYPES ][XFS_QM_TRANS_MAXDQS ]; };
fs/xfs/xfs_trans_dquot.c +10 -5
@@ -379,24 +379,29 @@ xfs_trans_mod_dquot(/* * Given an array of dqtrx structures, lock all the dquots associated and join -* them to the transaction, provided they have been modified. We know that the -* highest number of dquots of one type - usr, grp and prj - involved in a -* transaction is 3 so we don't need to make this very generic. +* them to the transaction, provided they have been modified. */ STATIC voidxfs_trans_dqlockedjoin (struct xfs_trans *tp,struct xfs_dqtrx *q) { +unsigned int i;ASSERT (q[0 ].qt_dquot !=NULL );if (q[1 ].qt_dquot ==NULL ) {xfs_dqlock (q[0 ].qt_dquot);xfs_trans_dqjoin (tp, q[0 ].qt_dquot); - }else { -ASSERT (XFS_QM_TRANS_MAXDQS ==2 ); + }else if (q[2 ].qt_dquot ==NULL ) {xfs_dqlock2 (q[0 ].qt_dquot, q[1 ].qt_dquot);xfs_trans_dqjoin (tp, q[0 ].qt_dquot);xfs_trans_dqjoin (tp, q[1 ].qt_dquot); + }else { +xfs_dqlockn (q); +for (i =0 ; i <XFS_QM_TRANS_MAXDQS ; i++) { +if (q[i].qt_dquot ==NULL ) +break ; +xfs_trans_dqjoin (tp, q[i].qt_dquot); + } } }
Re: [PATCH 2/7] xfs: Increase XFS_QM_TRANS_MAXDQS to 5
Looks good:
PATCH 3/7 xfs: Hold inode locks in xfs_ialloc
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_ialloc to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 12 +++++++++--- fs/xfs/xfs_qm.c | 4 +++- fs/xfs/xfs_symlink.c | 6 ++++-- 3 files changed, 16 insertions(+), 6 deletions(-)
fs/xfs/xfs_inode.c +9 -3
@@ -747,6 +747,8 @@ xfs_inode_inherit_flags2(/* * Initialise a newly allocated inode and return the in-core inode to the * caller locked exclusively. +* +* Caller is responsible for unlocking the inode manually upon return */ int xfs_init_new_inode(
@@ -873,7 +875,7 @@ xfs_init_new_inode(/* * Log the new values stuffed into the inode. */ -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, ip,0 );xfs_trans_log_inode (tp, ip, flags);/* now that we have an i_mode we can setup the inode structure */
@@ -1101,8 +1103,7 @@ xfs_create(* the transaction cancel unlocking dp so don't do it explicitly in the * error path. */ -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); - unlock_dp_on_error = false; +xfs_trans_ijoin (tp, dp,0 ); error =xfs_dir_createname (tp, dp, name, ip->i_ino, resblks -XFS_IALLOC_SPACE_RES (mp));
@@ -1151,6 +1152,8 @@ xfs_create(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -1162,6 +1165,7 @@ xfs_create(* transactions and deadlocks from xfs_inactive. */ if (ip) { +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_finish_inode_setup (ip);xfs_irele (ip); }
@@ -1247,6 +1251,7 @@ xfs_create_tmpfile(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -1258,6 +1263,7 @@ xfs_create_tmpfile(* transactions and deadlocks from xfs_inactive. */ if (ip) { +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_finish_inode_setup (ip);xfs_irele (ip); }
fs/xfs/xfs_qm.c +3 -1
@@ -836,8 +836,10 @@ xfs_qm_qino_alloc(ASSERT (xfs_is_shutdown (mp));xfs_alert (mp,"%s failed (error %d)!" , __func__, error); } -if (need_alloc) +if (need_alloc) { +xfs_iunlock (*ipp,XFS_ILOCK_EXCL );xfs_finish_inode_setup (*ipp); + }return error; }
fs/xfs/xfs_symlink.c +4 -2
@@ -172,8 +172,7 @@ xfs_symlink(* the transaction cancel unlocking dp so don't do it explicitly in the * error path. */ -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); - unlock_dp_on_error = false; +xfs_trans_ijoin (tp, dp,0 );/* * Also attach the dquot(s) to it, if applicable.
@@ -215,6 +214,8 @@ xfs_symlink(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -226,6 +227,7 @@ xfs_symlink(* transactions and deadlocks from xfs_inactive. */ if (ip) { +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_finish_inode_setup (ip);xfs_irele (ip); }
Re: [PATCH 3/7] xfs: Hold inode locks in xfs_ialloc
Looks good:
PATCH 4/7 xfs: Hold inode locks in xfs_trans_alloc_dir
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_trans_alloc_dir to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 14 ++++++++++++-- fs/xfs/xfs_trans.c | 9 +++++++-- 2 files changed, 19 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +12 -2
@@ -1368,10 +1368,15 @@ xfs_link(if (xfs_has_wsync (mp) ||xfs_has_dirsync (mp))xfs_trans_set_sync (tp); -return xfs_trans_commit (tp); + error =xfs_trans_commit (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); +return error; error_return:xfs_trans_cancel (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;
@@ -2781,15 +2786,20 @@ xfs_remove( error =xfs_trans_commit (tp);if (error) - goto std_return; + goto out_unlock;if (is_dir &&xfs_inode_is_filestream (ip))xfs_filestream_deassociate (ip); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:xfs_trans_cancel (tp); + out_unlock: +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL ); std_return:return error; }
fs/xfs/xfs_trans.c +7 -2
@@ -1430,6 +1430,8 @@ xfs_trans_alloc_ichange(* The caller must ensure that the on-disk dquots attached to this inode have * already been allocated and initialized. The ILOCKs will be dropped when the * transaction is committed or cancelled. +* +* Caller is responsible for unlocking the inodes manually upon return */ int xfs_trans_alloc_dir(
@@ -1460,8 +1462,8 @@ xfs_trans_alloc_dir(xfs_lock_two_inodes (dp,XFS_ILOCK_EXCL , ip,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); +xfs_trans_ijoin (tp, ip,0 ); error =xfs_qm_dqattach_locked (dp, false);if (error) {
@@ -1484,6 +1486,9 @@ xfs_trans_alloc_dir(if (error == -EDQUOT || error == -ENOSPC ) {if (!retried) {xfs_trans_cancel (tp); +xfs_iunlock (dp,XFS_ILOCK_EXCL ); +if (dp != ip) +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_blockgc_free_quota (dp,0 ); retried = true; goto retry;
Re: [PATCH 4/7] xfs: Hold inode locks in xfs_trans_alloc_dir
Looks good:
PATCH 5/7 xfs: Hold inode locks in xfs_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_rename to hold all inode locks across a rename operation We will need this later when we add parent pointers
--- fs/xfs/xfs_inode.c | 45 +++++++++++++++++++++++++++++++++------------ 1 file changed, 33 insertions(+), 12 deletions(-)
fs/xfs/xfs_inode.c +33 -12
@@ -2804,6 +2804,21 @@ xfs_remove(return error; } +static inline void +xfs_iunlock_rename ( +struct xfs_inode **i_tab, +int num_inodes) +{ +int i; + +for (i = num_inodes -1 ; i >=0 ; i--) { +/* Skip duplicate inodes if src and target dps are the same */ +if (!i_tab[i] || (i >0 && i_tab[i] == i_tab[i -1 ])) +continue ; +xfs_iunlock (i_tab[i],XFS_ILOCK_EXCL ); + } +} +/* * Enter all inodes for a rename transaction into a sorted array. */
@@ -3113,8 +3128,10 @@ xfs_rename(* Attach the dquots to the inodes */ error =xfs_qm_vop_rename_dqattach (inodes); -if (error) - goto out_trans_cancel; +if (error) { +xfs_trans_cancel (tp); + goto out_release_wip; + }/* * Lock all the participating inodes. Depending upon whether
@@ -3125,18 +3142,16 @@ xfs_rename(xfs_lock_inodes (inodes, num_inodes,XFS_ILOCK_EXCL );/* -* Join all the inodes to the transaction. From this point on, -* we can rely on either trans_commit or trans_cancel to unlock -* them. +* Join all the inodes to the transaction. */ -xfs_trans_ijoin (tp, src_dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, src_dp,0 );if (new_parent) -xfs_trans_ijoin (tp, target_dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, src_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_dp,0 ); +xfs_trans_ijoin (tp, src_ip,0 );if (target_ip) -xfs_trans_ijoin (tp, target_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_ip,0 );if (wip) -xfs_trans_ijoin (tp, wip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, wip,0 );/* * If we are using project inheritance, we only allow renames
@@ -3150,10 +3165,13 @@ xfs_rename( }/* RENAME_EXCHANGE is unique from here on. */ -if (flags &RENAME_EXCHANGE ) -return xfs_cross_rename (tp, src_dp, src_name, src_ip, +if (flags &RENAME_EXCHANGE ) { + error =xfs_cross_rename (tp, src_dp, src_name, src_ip, target_dp, target_name, target_ip, spaceres); +xfs_iunlock_rename (inodes, num_inodes); +return error; + }/* * Try to reserve quota to handle an expansion of the target directory.
@@ -3167,6 +3185,7 @@ xfs_rename(if (error == -EDQUOT || error == -ENOSPC ) {if (!retried) {xfs_trans_cancel (tp); +xfs_iunlock_rename (inodes, num_inodes);xfs_blockgc_free_quota (target_dp,0 ); retried = true; goto retry;
@@ -3393,12 +3412,14 @@ xfs_rename(xfs_dir_update_hook (src_dp, wip,1 , src_name); error =xfs_finish_rename (tp); +xfs_iunlock_rename (inodes, num_inodes);if (wip)xfs_irele (wip);return error; out_trans_cancel:xfs_trans_cancel (tp); +xfs_iunlock_rename (inodes, num_inodes); out_release_wip:if (wip)xfs_irele (wip);
Re: [PATCH 5/7] xfs: Hold inode locks in xfs_rename
Looks good:
PATCH 6/7 xfs: don't pick up IOLOCK during rmapbt repair scan
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now that we've fixed the directory operations to hold the ILOCK until they're finished with rmapbt updates for directory shape changes, we no longer need to take this lock when scanning directories for rmapbt records.
--- fs/xfs/scrub/rmap_repair.c | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-)
fs/xfs/scrub/rmap_repair.c +1 -15
@@ -578,23 +578,9 @@ xrep_rmap_scan_inode(struct xrep_rmap *rr, struct xfs_inode *ip) { - unsigned int lock_mode = 0; + unsigned int lock_mode = xrep_rmap_scan_ilock(ip);int error; -/* -* Directory updates (create/link/unlink/rename) drop the directory's -* ILOCK before finishing any rmapbt updates associated with directory -* shape changes. For this scan to coordinate correctly with the live -* update hook, we must take the only lock (i_rwsem) that is held all -* the way to dir op completion. This will get fixed by the parent -* pointer patchset. -*/ -if (S_ISDIR (VFS_I (ip)->i_mode)) { - lock_mode =XFS_IOLOCK_SHARED ; -xfs_ilock (ip, lock_mode); - } - lock_mode |=xrep_rmap_scan_ilock (ip); -/* Check the data fork. */ error =xrep_rmap_scan_ifork (rr, ip,XFS_DATA_FORK );if (error)
Re: [PATCH 6/7] xfs: don't pick up IOLOCK during rmapbt repair scan
Looks good:
PATCH 7/7 xfs: unlock new repair tempfiles after creation
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
After creation, drop the ILOCK on temporary files that have been created to stage a repair.
--- fs/xfs/scrub/tempfile.c | 2 ++ 1 file changed, 2 insertions(+)
fs/xfs/scrub/tempfile.c +2 -0
@@ -153,6 +153,7 @@ xrep_tempfile_create(xfs_qm_dqrele (pdqp);/* Finish setting up the incore / vfs context. */ +xfs_iunlock (sc->tempip,XFS_ILOCK_EXCL );xfs_setup_iops (sc->tempip);xfs_finish_inode_setup (sc->tempip);
@@ -168,6 +169,7 @@ xrep_tempfile_create(* transactions and deadlocks from xfs_inactive. */ if (sc->tempip) { +xfs_iunlock (sc->tempip,XFS_ILOCK_EXCL );xfs_finish_inode_setup (sc->tempip);xchk_irele (sc, sc->tempip); }
Re: [PATCH 7/7] xfs: unlock new repair tempfiles after creation
Looks good:
PATCH 3/9 xfs: shrink struct xfs_da_args
In reply to Darrick J. WongHi all,
Let's clean out some unused flags and fields from struct xfs_da_args.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=shrink-dirattr-args
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=shrink-dirattr-args
---
Commits in this patchset:
* xfs: remove XFS_DA_OP_REMOVE
* xfs: remove XFS_DA_OP_NOTIME
* xfs: rename xfs_da_args.attr_flags
* xfs: rearrange xfs_da_args a bit to use less space
--- fs/xfs/libxfs/xfs_attr.c | 9 ++++----- fs/xfs/libxfs/xfs_attr.h | 1 - fs/xfs/libxfs/xfs_da_btree.h | 30 ++++++++++++++---------------- fs/xfs/scrub/attr.c | 1 - fs/xfs/scrub/attr_repair.c | 2 +- fs/xfs/xfs_ioctl.c | 6 +++--- fs/xfs/xfs_trace.h | 6 +++--- fs/xfs/xfs_xattr.c | 2 +- 8 files changed, 26 insertions(+), 31 deletions(-)
PATCH 1/4 xfs: remove XFS_DA_OP_REMOVE
From: Darrick J. Wong <[email protected]>
Nobody checks this flag, so get rid of it.
--- fs/xfs/libxfs/xfs_attr.h | 1 - fs/xfs/libxfs/xfs_da_btree.h | 6 ++---- 2 files changed, 2 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_attr.h +0 -1
@@ -590,7 +590,6 @@ xfs_attr_init_add_state(struct xfs_da_args *args)static inline enum xfs_delattr_state xfs_attr_init_remove_state (struct xfs_da_args *args) { - args->op_flags |=XFS_DA_OP_REMOVE ;if (xfs_attr_is_shortform (args->dp))return XFS_DAS_SF_REMOVE ;if (xfs_attr_is_leaf (args->dp))
fs/xfs/libxfs/xfs_da_btree.h +2 -4
@@ -91,9 +91,8 @@ typedef struct xfs_da_args {#define XFS_DA_OP_OKNOENT (1u << 3)/* lookup op, ENOENT ok, else die */ #define XFS_DA_OP_CILOOKUP (1u << 4)/* lookup returns CI name if found */ #define XFS_DA_OP_NOTIME (1u << 5)/* don't update inode timestamps */ -#define XFS_DA_OP_REMOVE (1u << 6)/* this is a remove operation */ -#define XFS_DA_OP_RECOVERY (1u << 7)/* Log recovery operation */ -#define XFS_DA_OP_LOGGED (1u << 8)/* Use intent items to track op */ +#define XFS_DA_OP_RECOVERY (1u << 6)/* Log recovery operation */ +#define XFS_DA_OP_LOGGED (1u << 7)/* Use intent items to track op */ #define XFS_DA_OP_FLAGS \ { XFS_DA_OP_JUSTCHECK, "JUSTCHECK" }, \
@@ -102,7 +101,6 @@ typedef struct xfs_da_args { { XFS_DA_OP_OKNOENT, "OKNOENT" }, \ { XFS_DA_OP_CILOOKUP, "CILOOKUP" }, \ { XFS_DA_OP_NOTIME, "NOTIME" }, \ - { XFS_DA_OP_REMOVE, "REMOVE" }, \ { XFS_DA_OP_RECOVERY, "RECOVERY" }, \ { XFS_DA_OP_LOGGED, "LOGGED" }
Re: [PATCH 1/4] xfs: remove XFS_DA_OP_REMOVE
Looks good:
PATCH 2/4 xfs: remove XFS_DA_OP_NOTIME
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The only user of this flag sets it prior to an xfs_attr_get_ilocked call, which doesn't update anything. Get rid of the flag.
--- fs/xfs/libxfs/xfs_attr.c | 5 ++--- fs/xfs/libxfs/xfs_da_btree.h | 6 ++---- fs/xfs/scrub/attr.c | 1 - 3 files changed, 4 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_attr.c +2 -3
@@ -365,7 +365,7 @@ xfs_attr_try_sf_addname(* Commit the shortform mods, and we're done. * NOTE: this is also the error path (EEXIST, etc). */ -if (!error && !(args->op_flags &XFS_DA_OP_NOTIME )) +if (!error)xfs_trans_ichgtime (args->trans, dp,XFS_ICHGTIME_CHG );if (xfs_has_wsync (dp->i_mount))
@@ -1033,8 +1033,7 @@ xfs_attr_set(if (xfs_has_wsync (mp))xfs_trans_set_sync (args->trans); -if (!(args->op_flags &XFS_DA_OP_NOTIME )) -xfs_trans_ichgtime (args->trans, dp,XFS_ICHGTIME_CHG ); +xfs_trans_ichgtime (args->trans, dp,XFS_ICHGTIME_CHG );/* * Commit the last in the sequence of transactions.
fs/xfs/libxfs/xfs_da_btree.h +2 -4
@@ -90,9 +90,8 @@ typedef struct xfs_da_args {#define XFS_DA_OP_ADDNAME (1u << 2)/* this is an add operation */ #define XFS_DA_OP_OKNOENT (1u << 3)/* lookup op, ENOENT ok, else die */ #define XFS_DA_OP_CILOOKUP (1u << 4)/* lookup returns CI name if found */ -#define XFS_DA_OP_NOTIME (1u << 5)/* don't update inode timestamps */ -#define XFS_DA_OP_RECOVERY (1u << 6)/* Log recovery operation */ -#define XFS_DA_OP_LOGGED (1u << 7)/* Use intent items to track op */ +#define XFS_DA_OP_RECOVERY (1u << 5)/* Log recovery operation */ +#define XFS_DA_OP_LOGGED (1u << 6)/* Use intent items to track op */ #define XFS_DA_OP_FLAGS \ { XFS_DA_OP_JUSTCHECK, "JUSTCHECK" }, \
@@ -100,7 +99,6 @@ typedef struct xfs_da_args { { XFS_DA_OP_ADDNAME, "ADDNAME" }, \ { XFS_DA_OP_OKNOENT, "OKNOENT" }, \ { XFS_DA_OP_CILOOKUP, "CILOOKUP" }, \ - { XFS_DA_OP_NOTIME, "NOTIME" }, \ { XFS_DA_OP_RECOVERY, "RECOVERY" }, \ { XFS_DA_OP_LOGGED, "LOGGED" }
fs/xfs/scrub/attr.c +0 -1
@@ -173,7 +173,6 @@ xchk_xattr_actor( void *priv) {struct xfs_da_args args = { - .op_flags =XFS_DA_OP_NOTIME , .attr_filter = attr_flags &XFS_ATTR_NSP_ONDISK_MASK , .geo = sc->mp->m_attr_geo, .whichfork =XFS_ATTR_FORK ,
Re: [PATCH 2/4] xfs: remove XFS_DA_OP_NOTIME
Looks good:
PATCH 3/4 xfs: rename xfs_da_args.attr_flags
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
This field only ever contains XATTR_{CREATE,REPLACE}, so let's change the name of the field to make the field and its values consistent.
--- fs/xfs/libxfs/xfs_attr.c | 4 ++-- fs/xfs/libxfs/xfs_da_btree.h | 2 +- fs/xfs/scrub/attr_repair.c | 2 +- fs/xfs/xfs_ioctl.c | 6 +++--- fs/xfs/xfs_trace.h | 6 +++--- fs/xfs/xfs_xattr.c | 2 +- 6 files changed, 11 insertions(+), 11 deletions(-)
fs/xfs/libxfs/xfs_attr.c +2 -2
@@ -1008,7 +1008,7 @@ xfs_attr_set( }/* Pure create fails if the attr already exists */ -if (args->attr_flags &XATTR_CREATE ) +if (args->xattr_flags &XATTR_CREATE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTRI_OP_FLAGS_REPLACE );break ;
@@ -1018,7 +1018,7 @@ xfs_attr_set( goto out_trans_cancel;/* Pure replace fails if no existing attr to replace. */ -if (args->attr_flags &XATTR_REPLACE ) +if (args->xattr_flags &XATTR_REPLACE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTRI_OP_FLAGS_SET );break ;
fs/xfs/libxfs/xfs_da_btree.h +1 -1
@@ -60,7 +60,7 @@ typedef struct xfs_da_args {void *value;/* set of bytes (maybe contain NULLs) */ int valuelen;/* length of value */ unsigned int attr_filter;/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ -unsigned int attr_flags;/* XATTR_{CREATE,REPLACE} */ +unsigned int xattr_flags;/* XATTR_{CREATE,REPLACE} */ xfs_dahash_t hashval;/* hash value of name */ xfs_ino_t inumber;/* input/output inode number */ struct xfs_inode *dp;/* directory inode to manipulate */
fs/xfs/scrub/attr_repair.c +1 -1
@@ -557,7 +557,7 @@ xrep_xattr_insert_rec(struct xfs_da_args args = { .dp = rx->sc->tempip, .attr_filter = key->flags, - .attr_flags =XATTR_CREATE , + .xattr_flags =XATTR_CREATE , .namelen = key->namelen, .valuelen = key->valuelen, .owner = rx->sc->ip->i_ino,
fs/xfs/xfs_ioctl.c +3 -3
@@ -362,7 +362,7 @@ xfs_attr_filter( }static unsigned int -xfs_attr_flags ( +xfs_xattr_flags (u32 ioc_flags) {if (ioc_flags &XFS_IOC_ATTR_CREATE )
@@ -476,7 +476,7 @@ xfs_attrmulti_attr_get(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .attr_flags =xfs_attr_flags (flags), + .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), .valuelen = *len,
@@ -510,7 +510,7 @@ xfs_attrmulti_attr_set(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .attr_flags =xfs_attr_flags (flags), + .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), };
fs/xfs/xfs_trace.h +3 -3
@@ -2000,7 +2000,7 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __field(int, valuelen) __field(xfs_dahash_t, hashval) __field(unsigned int, attr_filter) - __field(unsigned int, attr_flags) + __field(unsigned int, xattr_flags) __field(uint32_t, op_flags) ), TP_fast_assign(
@@ -2012,7 +2012,7 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->valuelen = args->valuelen; __entry->hashval = args->hashval; __entry->attr_filter = args->attr_filter; - __entry->attr_flags = args->attr_flags; + __entry->xattr_flags = args->xattr_flags; __entry->op_flags = args->op_flags; ), TP_printk("dev %d:%d ino 0x%llx name %.*s namelen %d valuelen %d "
@@ -2026,7 +2026,7 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->hashval,__print_flags (__entry->attr_filter,"|" ,XFS_ATTR_FILTER_FLAGS ), -__print_flags (__entry->attr_flags,"|" , +__print_flags (__entry->xattr_flags,"|" , { XATTR_CREATE, "CREATE" }, { XATTR_REPLACE, "REPLACE" }), __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS))
fs/xfs/xfs_xattr.c +1 -1
@@ -124,7 +124,7 @@ xfs_xattr_set(const struct xattr_handler *handler,struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter = handler->flags, - .attr_flags = flags, + .xattr_flags = flags, .name = name, .namelen =strlen (name), .value = (void *)value,
Re: [PATCH 3/4] xfs: rename xfs_da_args.attr_flags
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This field only ever contains XATTR_{CREATE,REPLACE}, so let's change the name of the field to make the field and its values consistent.
So, these flags only get passed to xfs_attr_set through xfs_attr_change and xfs_attr_setname, which means we should probably just pass them directly as in my patch (against your whole stack) below.
Also I suspect we should do an audit of all the internal callers if they should ever be replace an existing attr, as I guess most don't. (and xfs_attr_change really should be folded into xfs_attr_set, the split is confusing as hell).
fs/xfs/libxfs/xfs_attr.c +8 -6
@@ -1034,7 +1034,8 @@ xfs_attr_ensure_iext(*/ int xfs_attr_set ( -struct xfs_da_args *args) +struct xfs_da_args *args, +uint8_t xattr_flags) {struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;
@@ -1109,7 +1110,7 @@ xfs_attr_set( }/* Pure create fails if the attr already exists */ -if (args->xattr_flags &XATTR_CREATE ) +if (xattr_flags &XATTR_CREATE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTR_DEFER_REPLACE );break ;
@@ -1119,7 +1120,7 @@ xfs_attr_set( goto out_trans_cancel;/* Pure replace fails if no existing attr to replace. */ -if (args->xattr_flags &XATTR_REPLACE ) +if (xattr_flags &XATTR_REPLACE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTR_DEFER_SET );break ;
@@ -1155,7 +1156,7 @@ xfs_attr_set(* Ensure that the xattr structure maps @args->name to @args->value. * * The caller must have initialized @args, attached dquots, and must not hold -* any ILOCKs. Only XATTR_CREATE may be specified in @args->xattr_flags. +* any ILOCKs. Only XATTR_CREATE may be specified in @xattr_flags. * Reserved data blocks may be used if @rsvd is set. * * Returns -EEXIST if XATTR_CREATE was specified and the name already exists.
@@ -1163,6 +1164,7 @@ xfs_attr_set(int xfs_attr_setname (struct xfs_da_args *args, +uint8_t xattr_flags,bool rsvd) {struct xfs_inode *dp = args->dp;
@@ -1172,7 +1174,7 @@ xfs_attr_setname(int rmt_extents =0 ;int error, local; -ASSERT (!(args->xattr_flags &XATTR_REPLACE )); +ASSERT (!(xattr_flags & ~XATTR_CREATE ));ASSERT (!args->trans); args->total =xfs_attr_calc_size (args, &local);
@@ -1198,7 +1200,7 @@ xfs_attr_setname(switch (error) {case -EEXIST :/* Pure create fails if the attr already exists */ -if (args->xattr_flags &XATTR_CREATE ) +if (xattr_flags &XATTR_CREATE ) goto out_trans_cancel;if (args->attr_filter &XFS_ATTR_PARENT )xfs_attr_defer_parent (args,XFS_ATTR_DEFER_REPLACE );
fs/xfs/libxfs/xfs_attr.h +2 -2
@@ -550,7 +550,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); -int xfs_attr_set (struct xfs_da_args *args); +int xfs_attr_set (struct xfs_da_args *args,uint8_t xattr_flags);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);bool xfs_attr_check_namespace (unsigned int attr_flags);
@@ -560,7 +560,7 @@ int xfs_attr_calc_size(struct xfs_da_args *args, int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); -int xfs_attr_setname (struct xfs_da_args *args,bool rsvd); +int xfs_attr_setname (struct xfs_da_args *args,uint8_t xattr_flags,bool rsvd);int xfs_attr_removename (struct xfs_da_args *args,bool rsvd);/*
fs/xfs/libxfs/xfs_da_btree.h +0 -1
@@ -69,7 +69,6 @@ typedef struct xfs_da_args {uint8_t filetype;/* filetype of inode for directories */ uint8_t op_flags;/* operation flags */ uint8_t attr_filter;/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ -uint8_t xattr_flags;/* XATTR_{CREATE,REPLACE} */ short namelen;/* length of string (maybe no NULL) */ short new_namelen;/* length of new attr name */ xfs_dahash_t hashval;/* hash value of name */
fs/xfs/libxfs/xfs_parent.c +1 -1
@@ -355,7 +355,7 @@ xfs_parent_set(memset (scratch,0 ,sizeof (struct xfs_da_args ));xfs_parent_da_args_init (scratch,NULL , pptr, ip, owner, parent_name); -return xfs_attr_setname (scratch, true); +return xfs_attr_setname (scratch,0 , true); }/*
fs/xfs/scrub/attr_repair.c +1 -2
@@ -615,7 +615,6 @@ xrep_xattr_insert_rec(struct xfs_da_args args = { .dp = rx->sc->tempip, .attr_filter = key->flags, - .xattr_flags =XATTR_CREATE , .namelen = key->namelen, .valuelen = key->valuelen, .owner = rx->sc->ip->i_ino,
@@ -675,7 +674,7 @@ xrep_xattr_insert_rec(* use reserved blocks because we can abort the repair with ENOSPC. */ xfs_attr_sethash (&args); - error =xfs_attr_setname (&args, false); + error =xfs_attr_setname (&args,XATTR_CREATE , false);if (error == -EEXIST ) error =0 ;
fs/xfs/scrub/parent_repair.c +1 -1
@@ -1031,7 +1031,7 @@ xrep_parent_insert_xattr( rp->xattr_name, key->namelen, key->valuelen);xfs_attr_sethash (&args); -return xfs_attr_setname (&args, false); +return xfs_attr_setname (&args,0 , false); }/*
fs/xfs/xfs_acl.c +1 -1
@@ -203,7 +203,7 @@ __xfs_set_acl(struct inode *inode, struct posix_acl *acl, int type)xfs_acl_to_disk (args.value, acl); } - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,0 );kvfree (args.value);/*
fs/xfs/xfs_handle.c +1 -3
@@ -492,7 +492,6 @@ xfs_attrmulti_attr_get(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), .valuelen = *len,
@@ -526,7 +525,6 @@ xfs_attrmulti_attr_set(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), };
@@ -544,7 +542,7 @@ xfs_attrmulti_attr_set( args.valuelen = len; } - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,xfs_xattr_flags (flags));if (!error && (flags &XFS_IOC_ATTR_ROOT ))xfs_forget_acl (inode, name);kfree (args.value);
fs/xfs/xfs_iops.c +1 -1
@@ -64,7 +64,7 @@ xfs_initxattrs( .value = xattr->value, .valuelen = xattr->value_len, }; - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,0 );if (error <0 )break ; }
fs/xfs/xfs_trace.h +1 -6
@@ -2131,7 +2131,6 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __field(int, valuelen) __field(xfs_dahash_t, hashval) __field(unsigned int, attr_filter) - __field(unsigned int, xattr_flags)__field (uint32_t , op_flags) ), TP_fast_assign(
@@ -2143,11 +2142,10 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->valuelen = args->valuelen; __entry->hashval = args->hashval; __entry->attr_filter = args->attr_filter; - __entry->xattr_flags = args->xattr_flags; __entry->op_flags = args->op_flags; ), TP_printk("dev %d:%d ino 0x%llx name %.*s namelen %d valuelen %d " - "hashval 0x%x filter %s flags %s op_flags %s", + "hashval 0x%x filter %s op_flags %s", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->namelen,
@@ -2157,9 +2155,6 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->hashval,__print_flags (__entry->attr_filter,"|" ,XFS_ATTR_FILTER_FLAGS ), -__print_flags (__entry->xattr_flags,"|" , - { XATTR_CREATE, "CREATE" }, - { XATTR_REPLACE, "REPLACE" }), __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS)) )
fs/xfs/xfs_xattr.c +4 -4
@@ -80,7 +80,8 @@ xfs_attr_want_log_assist(*/ int xfs_attr_change ( -struct xfs_da_args *args) +struct xfs_da_args *args, +uint8_t xattr_flags) {struct xfs_mount *mp = args->dp->i_mount;int error;
@@ -95,7 +96,7 @@ xfs_attr_change( args->op_flags |=XFS_DA_OP_LOGGED ; } -return xfs_attr_set (args); +return xfs_attr_set (args, xattr_flags); }
@@ -131,7 +132,6 @@ xfs_xattr_set(const struct xattr_handler *handler,struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter = handler->flags, - .xattr_flags = flags, .name = name, .namelen =strlen (name), .value = (void *)value,
@@ -139,7 +139,7 @@ xfs_xattr_set(const struct xattr_handler *handler, };int error; - error =xfs_attr_change (&args); + error =xfs_attr_change (&args, flags);if (!error && (handler->flags &XFS_ATTR_ROOT ))xfs_forget_acl (inode, name);return error;
fs/xfs/xfs_xattr.h +1 -1
@@ -6,7 +6,7 @@#ifndef __XFS_XATTR_H__#define __XFS_XATTR_H__ -int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_change (struct xfs_da_args *args,uint8_t xattr_flags);int xfs_attr_grab_log_assist (struct xfs_mount *mp);void xfs_attr_rele_log_assist (struct xfs_mount *mp);
Re: [PATCH 3/4] xfs: rename xfs_da_args.attr_flags
Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This field only ever contains XATTR_{CREATE,REPLACE}, so let's change the name of the field to make the field and its values consistent.
So, these flags only get passed to
xfs_attr_setthroughxfs_attr_changeandxfs_attr_setname, which means we should probably just pass them directly as in my patch (against your whole stack) below.
Want me to reflow this through the tree, or just tack it on the end after (perhaps?) "xfs: fix corruptions in the directory tree" ?
Quoted 4 lines
Also I suspect we should do an audit of all the internal callers if they should ever be replace an existing attr, as I guess most don't. (and
xfs_attr_changereally should be folded intoxfs_attr_set, the split is confusing as hell).
I imagine a lot of the security stuff with magic xattrs probably only ever creates xattrs, but I would bet that some of these subsystems actually *want* the upsert behavior -- "the frob for this file should be $foo, make it so".
--D
Quoted 292 lines
fs/xfs/libxfs/xfs_attr.c +8
-6@@ -1034,7 +1034,8 @@ xfs_attr_ensure_iext(*/ int xfs_attr_set ( -struct xfs_da_args *args) +struct xfs_da_args *args, +uint8_t xattr_flags) {struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;@@ -1109,7 +1110,7 @@ xfs_attr_set( }/* Pure create fails if the attr already exists */ -if (args->xattr_flags &XATTR_CREATE ) +if (xattr_flags &XATTR_CREATE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTR_DEFER_REPLACE );break ;@@ -1119,7 +1120,7 @@ xfs_attr_set( goto out_trans_cancel;/* Pure replace fails if no existing attr to replace. */ -if (args->xattr_flags &XATTR_REPLACE ) +if (xattr_flags &XATTR_REPLACE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTR_DEFER_SET );break ;@@ -1155,7 +1156,7 @@ xfs_attr_set(* Ensure that the xattr structure maps @args->name to @args->value. * * The caller must have initialized @args, attached dquots, and must not hold -* any ILOCKs. Only XATTR_CREATE may be specified in @args->xattr_flags. +* any ILOCKs. Only XATTR_CREATE may be specified in @xattr_flags. * Reserved data blocks may be used if @rsvd is set. * * Returns -EEXIST if XATTR_CREATE was specified and the name already exists. @@ -1163,6 +1164,7 @@ xfs_attr_set(int xfs_attr_setname (struct xfs_da_args *args, +uint8_t xattr_flags,bool rsvd) {struct xfs_inode *dp = args->dp;@@ -1172,7 +1174,7 @@ xfs_attr_setname(int rmt_extents =0 ;int error, local; -ASSERT (!(args->xattr_flags &XATTR_REPLACE )); +ASSERT (!(xattr_flags & ~XATTR_CREATE ));ASSERT (!args->trans); args->total =xfs_attr_calc_size (args, &local);@@ -1198,7 +1200,7 @@ xfs_attr_setname(switch (error) {case -EEXIST :/* Pure create fails if the attr already exists */ -if (args->xattr_flags &XATTR_CREATE ) +if (xattr_flags &XATTR_CREATE ) goto out_trans_cancel;if (args->attr_filter &XFS_ATTR_PARENT )xfs_attr_defer_parent (args,XFS_ATTR_DEFER_REPLACE );fs/xfs/libxfs/xfs_attr.h +2
-2@@ -550,7 +550,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); -int xfs_attr_set (struct xfs_da_args *args); +int xfs_attr_set (struct xfs_da_args *args,uint8_t xattr_flags);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);bool xfs_attr_check_namespace (unsigned int attr_flags);@@ -560,7 +560,7 @@ int xfs_attr_calc_size(struct xfs_da_args *args, int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); -int xfs_attr_setname (struct xfs_da_args *args,bool rsvd); +int xfs_attr_setname (struct xfs_da_args *args,uint8_t xattr_flags,bool rsvd);int xfs_attr_removename (struct xfs_da_args *args,bool rsvd);/* fs/xfs/libxfs/xfs_da_btree.h +0
-1@@ -69,7 +69,6 @@ typedef struct xfs_da_args {uint8_t filetype;/* filetype of inode for directories */ uint8_t op_flags;/* operation flags */ uint8_t attr_filter;/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ -uint8_t xattr_flags;/* XATTR_{CREATE,REPLACE} */ short namelen;/* length of string (maybe no NULL) */ short new_namelen;/* length of new attr name */ xfs_dahash_t hashval;/* hash value of name */ fs/xfs/libxfs/xfs_parent.c +1
-1@@ -355,7 +355,7 @@ xfs_parent_set(memset (scratch,0 ,sizeof (struct xfs_da_args ));xfs_parent_da_args_init (scratch,NULL , pptr, ip, owner, parent_name); -return xfs_attr_setname (scratch, true); +return xfs_attr_setname (scratch,0 , true); }/* fs/xfs/scrub/attr_repair.c +1
-2@@ -615,7 +615,6 @@ xrep_xattr_insert_rec(struct xfs_da_args args = { .dp = rx->sc->tempip, .attr_filter = key->flags, - .xattr_flags =XATTR_CREATE , .namelen = key->namelen, .valuelen = key->valuelen, .owner = rx->sc->ip->i_ino,@@ -675,7 +674,7 @@ xrep_xattr_insert_rec(* use reserved blocks because we can abort the repair with ENOSPC. */ xfs_attr_sethash (&args); - error =xfs_attr_setname (&args, false); + error =xfs_attr_setname (&args,XATTR_CREATE , false);if (error == -EEXIST ) error =0 ;fs/xfs/scrub/parent_repair.c +1
-1@@ -1031,7 +1031,7 @@ xrep_parent_insert_xattr( rp->xattr_name, key->namelen, key->valuelen);xfs_attr_sethash (&args); -return xfs_attr_setname (&args, false); +return xfs_attr_setname (&args,0 , false); }/* fs/xfs/xfs_acl.c +1
-1@@ -203,7 +203,7 @@ __xfs_set_acl(struct inode *inode, struct posix_acl *acl, int type)xfs_acl_to_disk (args.value, acl); } - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,0 );kvfree (args.value);/* fs/xfs/xfs_handle.c +1
-3@@ -492,7 +492,6 @@ xfs_attrmulti_attr_get(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), .valuelen = *len,@@ -526,7 +525,6 @@ xfs_attrmulti_attr_set(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), };@@ -544,7 +542,7 @@ xfs_attrmulti_attr_set( args.valuelen = len; } - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,xfs_xattr_flags (flags));if (!error && (flags &XFS_IOC_ATTR_ROOT ))xfs_forget_acl (inode, name);kfree (args.value);fs/xfs/xfs_iops.c +1
-1@@ -64,7 +64,7 @@ xfs_initxattrs( .value = xattr->value, .valuelen = xattr->value_len, }; - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,0 );if (error <0 )break ; }fs/xfs/xfs_trace.h +1
-6@@ -2131,7 +2131,6 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __field(int, valuelen) __field(xfs_dahash_t, hashval) __field(unsigned int, attr_filter) - __field(unsigned int, xattr_flags)__field (uint32_t , op_flags) ), TP_fast_assign(@@ -2143,11 +2142,10 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->valuelen = args->valuelen; __entry->hashval = args->hashval; __entry->attr_filter = args->attr_filter; - __entry->xattr_flags = args->xattr_flags; __entry->op_flags = args->op_flags; ), TP_printk("dev %d:%d ino 0x%llx name %.*s namelen %d valuelen %d " - "hashval 0x%x filter %s flags %s op_flags %s", + "hashval 0x%x filter %s op_flags %s", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->namelen,@@ -2157,9 +2155,6 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->hashval,__print_flags (__entry->attr_filter,"|" ,XFS_ATTR_FILTER_FLAGS ), -__print_flags (__entry->xattr_flags,"|" , - { XATTR_CREATE, "CREATE" }, - { XATTR_REPLACE, "REPLACE" }), __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS)) )fs/xfs/xfs_xattr.c +4
-4@@ -80,7 +80,8 @@ xfs_attr_want_log_assist(*/ int xfs_attr_change ( -struct xfs_da_args *args) +struct xfs_da_args *args, +uint8_t xattr_flags) {struct xfs_mount *mp = args->dp->i_mount;int error;@@ -95,7 +96,7 @@ xfs_attr_change( args->op_flags |=XFS_DA_OP_LOGGED ; } -return xfs_attr_set (args); +return xfs_attr_set (args, xattr_flags); }@@ -131,7 +132,6 @@ xfs_xattr_set(const struct xattr_handler *handler,struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter = handler->flags, - .xattr_flags = flags, .name = name, .namelen =strlen (name), .value = (void *)value,@@ -139,7 +139,7 @@ xfs_xattr_set(const struct xattr_handler *handler, };int error; - error =xfs_attr_change (&args); + error =xfs_attr_change (&args, flags);if (!error && (handler->flags &XFS_ATTR_ROOT ))xfs_forget_acl (inode, name);return error;fs/xfs/xfs_xattr.h +1
-1@@ -6,7 +6,7 @@#ifndef __XFS_XATTR_H__#define __XFS_XATTR_H__ -int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_change (struct xfs_da_args *args,uint8_t xattr_flags);int xfs_attr_grab_log_assist (struct xfs_mount *mp);void xfs_attr_rele_log_assist (struct xfs_mount *mp);
Re: [PATCH 3/4] xfs: rename xfs_da_args.attr_flags
Quoted 7 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This field only ever contains XATTR_{CREATE,REPLACE}, so let's change the name of the field to make the field and its values consistent.
Quoted 6 lines
Quoted 3 lines
So, these flags only get passed to
xfs_attr_setthroughxfs_attr_changeandxfs_attr_setname, which means we should probably just pass them directly as in my patch (against your whole stack) below.Want me to reflow this through the tree, or just tack it on the end after (perhaps?) "xfs: fix corruptions in the directory tree" ?
Ugh, no, that got messy so I just tacked it on the end. :)
Also I changed the uint8_t parameter to int because the XATTR_* flags mostly come from the VFS and that's what it passes us in xattr_handler::set().
--D
Quoted 305 lines
Quoted 4 lines
Also I suspect we should do an audit of all the internal callers if they should ever be replace an existing attr, as I guess most don't. (and
xfs_attr_changereally should be folded intoxfs_attr_set, the split is confusing as hell).I imagine a lot of the security stuff with magic xattrs probably only ever creates xattrs, but I would bet that some of these subsystems actually *want* the upsert behavior -- "the frob for this file should be $foo, make it so".
--D
Quoted 292 lines
fs/xfs/libxfs/xfs_attr.c +8
-6@@ -1034,7 +1034,8 @@ xfs_attr_ensure_iext(*/ int xfs_attr_set ( -struct xfs_da_args *args) +struct xfs_da_args *args, +uint8_t xattr_flags) {struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;@@ -1109,7 +1110,7 @@ xfs_attr_set( }/* Pure create fails if the attr already exists */ -if (args->xattr_flags &XATTR_CREATE ) +if (xattr_flags &XATTR_CREATE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTR_DEFER_REPLACE );break ;@@ -1119,7 +1120,7 @@ xfs_attr_set( goto out_trans_cancel;/* Pure replace fails if no existing attr to replace. */ -if (args->xattr_flags &XATTR_REPLACE ) +if (xattr_flags &XATTR_REPLACE ) goto out_trans_cancel;xfs_attr_defer_add (args,XFS_ATTR_DEFER_SET );break ;@@ -1155,7 +1156,7 @@ xfs_attr_set(* Ensure that the xattr structure maps @args->name to @args->value. * * The caller must have initialized @args, attached dquots, and must not hold -* any ILOCKs. Only XATTR_CREATE may be specified in @args->xattr_flags. +* any ILOCKs. Only XATTR_CREATE may be specified in @xattr_flags. * Reserved data blocks may be used if @rsvd is set. * * Returns -EEXIST if XATTR_CREATE was specified and the name already exists. @@ -1163,6 +1164,7 @@ xfs_attr_set(int xfs_attr_setname (struct xfs_da_args *args, +uint8_t xattr_flags,bool rsvd) {struct xfs_inode *dp = args->dp;@@ -1172,7 +1174,7 @@ xfs_attr_setname(int rmt_extents =0 ;int error, local; -ASSERT (!(args->xattr_flags &XATTR_REPLACE )); +ASSERT (!(xattr_flags & ~XATTR_CREATE ));ASSERT (!args->trans); args->total =xfs_attr_calc_size (args, &local);@@ -1198,7 +1200,7 @@ xfs_attr_setname(switch (error) {case -EEXIST :/* Pure create fails if the attr already exists */ -if (args->xattr_flags &XATTR_CREATE ) +if (xattr_flags &XATTR_CREATE ) goto out_trans_cancel;if (args->attr_filter &XFS_ATTR_PARENT )xfs_attr_defer_parent (args,XFS_ATTR_DEFER_REPLACE );fs/xfs/libxfs/xfs_attr.h +2
-2@@ -550,7 +550,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); -int xfs_attr_set (struct xfs_da_args *args); +int xfs_attr_set (struct xfs_da_args *args,uint8_t xattr_flags);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);bool xfs_attr_check_namespace (unsigned int attr_flags);@@ -560,7 +560,7 @@ int xfs_attr_calc_size(struct xfs_da_args *args, int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); -int xfs_attr_setname (struct xfs_da_args *args,bool rsvd); +int xfs_attr_setname (struct xfs_da_args *args,uint8_t xattr_flags,bool rsvd);int xfs_attr_removename (struct xfs_da_args *args,bool rsvd);/* fs/xfs/libxfs/xfs_da_btree.h +0
-1@@ -69,7 +69,6 @@ typedef struct xfs_da_args {uint8_t filetype;/* filetype of inode for directories */ uint8_t op_flags;/* operation flags */ uint8_t attr_filter;/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ -uint8_t xattr_flags;/* XATTR_{CREATE,REPLACE} */ short namelen;/* length of string (maybe no NULL) */ short new_namelen;/* length of new attr name */ xfs_dahash_t hashval;/* hash value of name */ fs/xfs/libxfs/xfs_parent.c +1
-1@@ -355,7 +355,7 @@ xfs_parent_set(memset (scratch,0 ,sizeof (struct xfs_da_args ));xfs_parent_da_args_init (scratch,NULL , pptr, ip, owner, parent_name); -return xfs_attr_setname (scratch, true); +return xfs_attr_setname (scratch,0 , true); }/* fs/xfs/scrub/attr_repair.c +1
-2@@ -615,7 +615,6 @@ xrep_xattr_insert_rec(struct xfs_da_args args = { .dp = rx->sc->tempip, .attr_filter = key->flags, - .xattr_flags =XATTR_CREATE , .namelen = key->namelen, .valuelen = key->valuelen, .owner = rx->sc->ip->i_ino,@@ -675,7 +674,7 @@ xrep_xattr_insert_rec(* use reserved blocks because we can abort the repair with ENOSPC. */ xfs_attr_sethash (&args); - error =xfs_attr_setname (&args, false); + error =xfs_attr_setname (&args,XATTR_CREATE , false);if (error == -EEXIST ) error =0 ;fs/xfs/scrub/parent_repair.c +1
-1@@ -1031,7 +1031,7 @@ xrep_parent_insert_xattr( rp->xattr_name, key->namelen, key->valuelen);xfs_attr_sethash (&args); -return xfs_attr_setname (&args, false); +return xfs_attr_setname (&args,0 , false); }/* fs/xfs/xfs_acl.c +1
-1@@ -203,7 +203,7 @@ __xfs_set_acl(struct inode *inode, struct posix_acl *acl, int type)xfs_acl_to_disk (args.value, acl); } - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,0 );kvfree (args.value);/* fs/xfs/xfs_handle.c +1
-3@@ -492,7 +492,6 @@ xfs_attrmulti_attr_get(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), .valuelen = *len,@@ -526,7 +525,6 @@ xfs_attrmulti_attr_set(struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter =xfs_attr_filter (flags), - .xattr_flags =xfs_xattr_flags (flags), .name = name, .namelen =strlen (name), };@@ -544,7 +542,7 @@ xfs_attrmulti_attr_set( args.valuelen = len; } - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,xfs_xattr_flags (flags));if (!error && (flags &XFS_IOC_ATTR_ROOT ))xfs_forget_acl (inode, name);kfree (args.value);fs/xfs/xfs_iops.c +1
-1@@ -64,7 +64,7 @@ xfs_initxattrs( .value = xattr->value, .valuelen = xattr->value_len, }; - error =xfs_attr_change (&args); + error =xfs_attr_change (&args,0 );if (error <0 )break ; }fs/xfs/xfs_trace.h +1
-6@@ -2131,7 +2131,6 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __field(int, valuelen) __field(xfs_dahash_t, hashval) __field(unsigned int, attr_filter) - __field(unsigned int, xattr_flags)__field (uint32_t , op_flags) ), TP_fast_assign(@@ -2143,11 +2142,10 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->valuelen = args->valuelen; __entry->hashval = args->hashval; __entry->attr_filter = args->attr_filter; - __entry->xattr_flags = args->xattr_flags; __entry->op_flags = args->op_flags; ), TP_printk("dev %d:%d ino 0x%llx name %.*s namelen %d valuelen %d " - "hashval 0x%x filter %s flags %s op_flags %s", + "hashval 0x%x filter %s op_flags %s", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->namelen,@@ -2157,9 +2155,6 @@ DECLARE_EVENT_CLASS(xfs_attr_class, __entry->hashval,__print_flags (__entry->attr_filter,"|" ,XFS_ATTR_FILTER_FLAGS ), -__print_flags (__entry->xattr_flags,"|" , - { XATTR_CREATE, "CREATE" }, - { XATTR_REPLACE, "REPLACE" }), __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS)) )fs/xfs/xfs_xattr.c +4
-4@@ -80,7 +80,8 @@ xfs_attr_want_log_assist(*/ int xfs_attr_change ( -struct xfs_da_args *args) +struct xfs_da_args *args, +uint8_t xattr_flags) {struct xfs_mount *mp = args->dp->i_mount;int error;@@ -95,7 +96,7 @@ xfs_attr_change( args->op_flags |=XFS_DA_OP_LOGGED ; } -return xfs_attr_set (args); +return xfs_attr_set (args, xattr_flags); }@@ -131,7 +132,6 @@ xfs_xattr_set(const struct xattr_handler *handler,struct xfs_da_args args = { .dp =XFS_I (inode), .attr_filter = handler->flags, - .xattr_flags = flags, .name = name, .namelen =strlen (name), .value = (void *)value,@@ -139,7 +139,7 @@ xfs_xattr_set(const struct xattr_handler *handler, };int error; - error =xfs_attr_change (&args); + error =xfs_attr_change (&args, flags);if (!error && (handler->flags &XFS_ATTR_ROOT ))xfs_forget_acl (inode, name);return error;fs/xfs/xfs_xattr.h +1
-1@@ -6,7 +6,7 @@#ifndef __XFS_XATTR_H__#define __XFS_XATTR_H__ -int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_change (struct xfs_da_args *args,uint8_t xattr_flags);int xfs_attr_grab_log_assist (struct xfs_mount *mp);void xfs_attr_rele_log_assist (struct xfs_mount *mp);
Re: [PATCH 3/4] xfs: rename xfs_da_args.attr_flags
In reply to Darrick J. WongQuoted 7 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This field only ever contains XATTR_{CREATE,REPLACE}, so let's change the name of the field to make the field and its values consistent.
Quoted 6 lines
Quoted 3 lines
So, these flags only get passed to
xfs_attr_setthroughxfs_attr_changeandxfs_attr_setname, which means we should probably just pass them directly as in my patch (against your whole stack) below.Want me to reflow this through the tree, or just tack it on the end after (perhaps?) "xfs: fix corruptions in the directory tree" ?
If it makes your life easier feel free to add it at the end.
Re: [PATCH 3/4] xfs: rename xfs_da_args.attr_flags
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This field only ever contains XATTR_{CREATE,REPLACE}, so let's change the name of the field to make the field and its values consistent.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
So, these flags only get passed to
xfs_attr_setthroughxfs_attr_changeandxfs_attr_setname, which means we should probably just pass them directly as in my patch (against your whole stack) below.Want me to reflow this through the tree, or just tack it on the end after (perhaps?) "xfs: fix corruptions in the directory tree" ?
If it makes your life easier feel free to add it at the end.
It does, and done!
--D
PATCH 4/4 xfs: rearrange xfs_da_args a bit to use less space
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
A few notes about struct xfs_da_args:
The XFS_ATTR_* flags only go up as far as XFS_ATTR_INCOMPLETE, which means that attr_filter could be a u8 field.
The XATTR_* flags only have two values, which means that xattr_flags could be shrunk to a u8.
I've reduced the number of XFS_DA_OP_* flags down to the point where op_flags would also fit into a u8.
filetype has 7 bytes of slack after it, which is wasteful.
namelen will never be greater than MAXNAMELEN, which is 256. This field could be reduced to a short.
Rearrange the fields in xfs_da_args to waste less space. This reduces the structure size from 136 bytes to 128. Later when we add extra fields to support parent pointer replacement, this will only bloat the structure to 144 bytes, instead of 168.
--- fs/xfs/libxfs/xfs_da_btree.h | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_da_btree.h +12 -10
@@ -54,17 +54,21 @@ enum xfs_dacmp {*/ typedef struct xfs_da_args { struct xfs_da_geometry *geo; /* da block geometry */ - const uint8_t *name; /* string (maybe not NULL terminated) */ - int namelen; /* length of string (maybe no NULL) */ - uint8_t filetype; /* filetype of inode for directories */ + const uint8_t *name; /* string (maybe not NULL terminated) */ void *value; /* set of bytes (maybe contain NULLs) */ - int valuelen; /* length of value */ - unsigned int attr_filter; /* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ - unsigned int xattr_flags; /* XATTR_{CREATE,REPLACE} */ - xfs_dahash_t hashval; /* hash value of name */ - xfs_ino_t inumber; /* input/output inode number */ struct xfs_inode *dp; /* directory inode to manipulate */ struct xfs_trans *trans; /* current trans (changes over time) */ + + xfs_ino_t inumber; /* input/output inode number */ + xfs_ino_t owner; /* inode that owns the dir/attr data */ + + int valuelen; /* length of value */ + uint8_t filetype; /* filetype of inode for directories */ + uint8_t op_flags; /* operation flags */ + uint8_t attr_filter; /* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ + uint8_t xattr_flags; /* XATTR_{CREATE,REPLACE} */ + short namelen; /* length of string (maybe no NULL) */ + xfs_dahash_t hashval; /* hash value of name */ xfs_extlen_t total; /* total blocks needed, for 1st bmap */ int whichfork; /* data or attribute fork */ xfs_dablk_t blkno; /* blkno of attr leaf of interest */
@@ -77,9 +81,7 @@ typedef struct xfs_da_args {xfs_dablk_t rmtblkno2;/* remote attr value starting blkno */ int rmtblkcnt2;/* remote attr value block count */ int rmtvaluelen2;/* remote attr value length in bytes */ -uint32_t op_flags;/* operation flags */ enum xfs_dacmp cmpresult;/* name compare result for lookups */ -xfs_ino_t owner;/* inode that owns the dir/attr data */ } xfs_da_args_t;/*
Re: [PATCH 4/4] xfs: rearrange xfs_da_args a bit to use less space
Quoted 15 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
A few notes about struct
xfs_da_args:The XFS_ATTR_* flags only go up as far as
XFS_ATTR_INCOMPLETE, which means thatattr_filtercould be a u8 field.The XATTR_* flags only have two values, which means that
xattr_flagscould be shrunk to a u8.I've reduced the number of XFS_DA_OP_* flags down to the point where
op_flagswould also fit into a u8.filetype has 7 bytes of slack after it, which is wasteful.
Quoted 7 lines
namelen will never be greater than MAXNAMELEN, which is 256. This field could be reduced to a short.
Rearrange the fields in
xfs_da_argsto waste less space. This reduces the structure size from 136 bytes to 128. Later when we add extra fields to support parent pointer replacement, this will only bloat the structure to 144 bytes, instead of 168.
Looks good:
Eventually we should probaly split this up, at lot of fields are used only by the attr set code, and a few less only by dir vs attr.
Re: [PATCH 4/4] xfs: rearrange xfs_da_args a bit to use less space
Quoted 24 lines · Christoph Hellwig
Quoted 15 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
A few notes about struct
xfs_da_args:The XFS_ATTR_* flags only go up as far as
XFS_ATTR_INCOMPLETE, which means thatattr_filtercould be a u8 field.The XATTR_* flags only have two values, which means that
xattr_flagscould be shrunk to a u8.I've reduced the number of XFS_DA_OP_* flags down to the point where
op_flagswould also fit into a u8.filetype has 7 bytes of slack after it, which is wasteful.
Quoted 7 lines
namelen will never be greater than MAXNAMELEN, which is 256. This field could be reduced to a short.
Rearrange the fields in
xfs_da_argsto waste less space. This reduces the structure size from 136 bytes to 128. Later when we add extra fields to support parent pointer replacement, this will only bloat the structure to 144 bytes, instead of 168.
Quoted 6 lines
Looks good:
Reviewed-by Christoph Hellwig <[email protected]>Eventually we should probaly split this up, at lot of fields are used only by the attr set code, and a few less only by dir vs attr.
Agreed, though we're veering dangerously close to object inheritance.
But it would be useful for code analysis if dir operations would pass in an xfs_dir_op structure containing a much smaller xfs_da_args and likewise for xattrs.
--D
PATCH 4/9 xfs: improve extended attribute validation
In reply to Darrick J. WongHi all,
Prior to introducing parent pointer extended attributes, let's spend some time cleaning up the attr code and strengthening the validation that it performs on attrs coming in from the disk.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=improve-attr-validation
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=improve-attr-validation
---
Commits in this patchset:
* xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
* xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery
* xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available
* xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2
* xfs: fix missing check for invalid attr flags
* xfs: restructure xfs_attr_complete_op a bit
* xfs: use helpers to extract xattr op from opflags
* xfs: validate recovered name buffers when recovering xattr items
* xfs: always set args->value in xfs_attri_item_recover
* xfs: use local variables for name and value length in _attri_commit_pass2
* xfs: refactor name/length checks in xfs_attri_validate
* xfs: enforce one namespace per attribute
--- fs/xfs/libxfs/xfs_attr.c | 41 ++++++++- fs/xfs/libxfs/xfs_attr.h | 9 ++ fs/xfs/libxfs/xfs_attr_leaf.c | 7 + fs/xfs/libxfs/xfs_da_format.h | 5 + fs/xfs/scrub/attr.c | 25 +++-- fs/xfs/scrub/attr_repair.c | 4 - fs/xfs/xfs_attr_item.c | 192 ++++++++++++++++++++++++++++++++--------- fs/xfs/xfs_attr_list.c | 18 +++- fs/xfs/xfs_mount.c | 16 +++ fs/xfs/xfs_mount.h | 6 + fs/xfs/xfs_xattr.c | 3 - 11 files changed, 258 insertions(+), 68 deletions(-)
PATCH 1/12 xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
From: Darrick J. Wong <[email protected]>
Christoph noticed that the xfs_attr_is_leaf in xfs_attr_get_ilocked can access the incore extent tree of the attr fork, but nothing in the xfs_attr_get path guarantees that the incore tree is actually loaded.
Most of the time it is, but seeing as xfs_attr_is_leaf ignores the return value of xfs_iext_get_extent I guess we've been making choices based on random stack contents and nobody's complained?
--- fs/xfs/libxfs/xfs_attr.c | 17 +++++++++++++++++ fs/xfs/xfs_attr_item.c | 42 ++++++++++++++++++++++++++++++++++++------ fs/xfs/xfs_attr_list.c | 7 +++++++ 3 files changed, 60 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_attr.c +17 -0
@@ -87,6 +87,8 @@ xfs_attr_is_leaf(struct xfs_iext_cursor icur;struct xfs_bmbt_irec imap; +ASSERT (!xfs_need_iread_extents (ifp)); +if (ifp->if_nextents !=1 || ifp->if_format !=XFS_DINODE_FMT_EXTENTS )return false;
@@ -224,11 +226,21 @@ intxfs_attr_get_ilocked ( struct xfs_da_args *args) { +int error; +xfs_assert_ilocked (args->dp,XFS_ILOCK_SHARED |XFS_ILOCK_EXCL );if (!xfs_inode_hasattr (args->dp))return -ENOATTR ; +/* +* The incore attr fork iext tree must be loaded for xfs_attr_is_leaf +* to work correctly. +*/ + error =xfs_iread_extents (args->trans, args->dp,XFS_ATTR_FORK ); +if (error) +return error; +if (args->dp->i_af.if_format ==XFS_DINODE_FMT_LOCAL )return xfs_attr_shortform_getvalue (args);if (xfs_attr_is_leaf (args->dp))
@@ -870,6 +882,11 @@ xfs_attr_lookup(return -ENOATTR ; } +/* Prerequisite for xfs_attr_is_leaf */ + error =xfs_iread_extents (args->trans, args->dp,XFS_ATTR_FORK ); +if (error) +return error; +if (xfs_attr_is_leaf (dp)) { error =xfs_attr_leaf_hasname (args, &bp);
fs/xfs/xfs_attr_item.c +36 -6
84 lines not on this page · raw message
fs/xfs/xfs_attr_list.c +7 -0
@@ -544,6 +544,7 @@ xfs_attr_list_ilocked( struct xfs_attr_list_context *context) {struct xfs_inode *dp = context->dp; +int error;xfs_assert_ilocked (dp,XFS_ILOCK_SHARED |XFS_ILOCK_EXCL );
@@ -554,6 +555,12 @@ xfs_attr_list_ilocked(return 0 ;if (dp->i_af.if_format ==XFS_DINODE_FMT_LOCAL )return xfs_attr_shortform_list (context); + +/* Prerequisite for xfs_attr_is_leaf */ + error =xfs_iread_extents (NULL , dp,XFS_ATTR_FORK ); +if (error) +return error; +if (xfs_attr_is_leaf (dp))return xfs_attr_leaf_list (context);return xfs_attr_node_list (context);
Re: [PATCH 01/12] xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
Quoted 9 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Christoph noticed that the
xfs_attr_is_leafinxfs_attr_get_ilockedcan access the incore extent tree of the attr fork, but nothing in thexfs_attr_getpath guarantees that the incore tree is actually loaded.Most of the time it is, but seeing as
xfs_attr_is_leafignores the return value ofxfs_iext_get_extentI guess we've been making choices based on random stack contents and nobody's complained?
Yes, I'm kinda puzzled.
Note that the dir code actually reads the extents in their is_leaf/is_block helpers. But given how the attr code is structured that would thread through a lot of code so it might not be worth it.
Re: [PATCH 01/12] xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Christoph noticed that the
xfs_attr_is_leafinxfs_attr_get_ilockedcan access the incore extent tree of the attr fork, but nothing in thexfs_attr_getpath guarantees that the incore tree is actually loaded.
Quoted 5 lines
Quoted 3 lines
Most of the time it is, but seeing as
xfs_attr_is_leafignores the return value ofxfs_iext_get_extentI guess we've been making choices based on random stack contents and nobody's complained?Yes, I'm kinda puzzled.
I suspect that most of the time we get lucky and *someone* has read in the attr fork or created it or whatever.
Quoted 3 lines
Note that the dir code actually reads the extents in their is_leaf/is_block helpers. But given how the attr code is structured that would thread through a lot of code so it might not be worth it.
<nod> But it would be more consistent...
Quoted 1 line
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
PATCH 2/12 xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The XFS_SB_FEAT_INCOMPAT_LOG_XATTRS feature bit protects a filesystem from old kernels that do not know how to recover extended attribute log intent items. Make this check mandatory instead of a debugging assert.
--- fs/xfs/xfs_attr_item.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
fs/xfs/xfs_attr_item.c +3 -2
@@ -469,6 +469,9 @@ xfs_attri_validate(unsigned int op = attrp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +if (!xfs_sb_version_haslogxattrs (&mp->m_sb)) +return false; +if (attrp->__pad !=0 )return false;
@@ -570,8 +573,6 @@ xfs_attri_recover_work(XFS_DA_OP_LOGGED ; args->owner = args->dp->i_ino; -ASSERT (xfs_sb_version_haslogxattrs (&mp->m_sb)); -switch (attr->xattri_op_flags) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :
Re: [PATCH 02/12] xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery
Looks good:
PATCH 3/12 xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Per reviewer request, use an OPSTATE flag (+ helpers) to decide if logged xattrs are enabled, instead of querying the xfs_sb.
--- fs/xfs/xfs_attr_item.c | 2 +- fs/xfs/xfs_mount.c | 16 ++++++++++++++++ fs/xfs/xfs_mount.h | 6 +++++- fs/xfs/xfs_xattr.c | 3 ++- 4 files changed, 24 insertions(+), 3 deletions(-)
fs/xfs/xfs_attr_item.c +1 -1
@@ -469,7 +469,7 @@ xfs_attri_validate(unsigned int op = attrp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; -if (!xfs_sb_version_haslogxattrs (&mp->m_sb)) +if (!xfs_is_using_logged_xattrs (mp))return false;if (attrp->__pad !=0 )
fs/xfs/xfs_mount.c +16 -0
@@ -230,6 +230,13 @@ xfs_readsb( mp->m_features |=xfs_sb_version_to_features (sbp);xfs_reinit_percpu_counters (mp); +/* +* If logged xattrs are enabled after log recovery finishes, then set +* the opstate so that log recovery will work properly. +*/ +if (xfs_sb_version_haslogxattrs (&mp->m_sb)) +xfs_set_using_logged_xattrs (mp); +/* no need to be quiet anymore, so reset the buf ops */ bp->b_ops = &xfs_sb_buf_ops;
@@ -828,6 +835,15 @@ xfs_mountfs( goto out_inodegc_shrinker; } +/* +* If logged xattrs are still enabled after log recovery finishes, then +* they'll be available until unmount. Otherwise, turn them off. +*/ +if (xfs_sb_version_haslogxattrs (&mp->m_sb)) +xfs_set_using_logged_xattrs (mp); +else +xfs_clear_using_logged_xattrs (mp); +/* Enable background inode inactivation workers. */ xfs_inodegc_start (mp);xfs_blockgc_start (mp);
fs/xfs/xfs_mount.h +5 -1
@@ -416,6 +416,8 @@ __XFS_HAS_FEAT(nouuid, NOUUID)#define XFS_OPSTATE_QUOTACHECK_RUNNING 10/* Do we want to clear log incompat flags? */ #define XFS_OPSTATE_UNSET_LOG_INCOMPAT 11 +/* Filesystem can use logged extended attributes */ +#define XFS_OPSTATE_USE_LARP 12#define __XFS_IS_OPSTATE (name,NAME ) \ static inline bool xfs_is_ ## name (struct xfs_mount *mp) \
@@ -444,6 +446,7 @@ __XFS_IS_OPSTATE(quotacheck_running, QUOTACHECK_RUNNING)# define xfs_is_quotacheck_running (mp) (false)#endif __XFS_IS_OPSTATE(done_with_log_incompat, UNSET_LOG_INCOMPAT) +__XFS_IS_OPSTATE(using_logged_xattrs, USE_LARP)static inline bool xfs_should_warn (struct xfs_mount *mp,long nr)
@@ -463,7 +466,8 @@ xfs_should_warn(struct xfs_mount *mp, long nr) { (1UL << XFS_OPSTATE_WARNED_SHRINK), "wshrink" }, \ { (1UL << XFS_OPSTATE_WARNED_LARP), "wlarp" }, \ { (1UL << XFS_OPSTATE_QUOTACHECK_RUNNING), "quotacheck" }, \ - { (1UL << XFS_OPSTATE_UNSET_LOG_INCOMPAT), "unset_log_incompat" } + { (1UL << XFS_OPSTATE_UNSET_LOG_INCOMPAT), "unset_log_incompat" }, \ + { (1UL << XFS_OPSTATE_USE_LARP), "logged_xattrs" }/* * Max and min values for mount-option defined I/O
fs/xfs/xfs_xattr.c +2 -1
@@ -31,7 +31,7 @@ xfs_attr_grab_log_assist(int error =0 ;/* xattr update log intent items are already enabled */ -if (xfs_sb_version_haslogxattrs (&mp->m_sb)) +if (xfs_is_using_logged_xattrs (mp))return 0 ;/*
@@ -48,6 +48,7 @@ xfs_attr_grab_log_assist( XFS_SB_FEAT_INCOMPAT_LOG_XATTRS);if (error)return error; +xfs_set_using_logged_xattrs (mp);xfs_warn_mount (mp,XFS_OPSTATE_WARNED_LARP ,"EXPERIMENTAL logged extended attributes feature in use. Use at your own risk!" );
Re: [PATCH 03/12] xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Per reviewer request, use an OPSTATE flag (+ helpers) to decide if logged xattrs are enabled, instead of querying the
xfs_sb.
Looks good:
PATCH 4/12 xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check that the number of recovered log iovecs is what is expected for the xattri opcode is expecting.
--- fs/xfs/xfs_attr_item.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
fs/xfs/xfs_attr_item.c +27 -0
@@ -737,6 +737,7 @@ xlog_recover_attri_commit_pass2(const void *attr_value =NULL ;const void *attr_name;size_t len; +unsigned int op; attri_formatp = item->ri_buf[0 ].i_addr; attr_name = item->ri_buf[1 ].i_addr;
@@ -755,6 +756,32 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } +/* Check the number of log iovecs makes sense for the op code. */ + op = attri_formatp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +switch (op) { +case XFS_ATTRI_OP_FLAGS_SET : +case XFS_ATTRI_OP_FLAGS_REPLACE : +/* Log item, attr name, attr value */ +if (item->ri_total !=3 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +break ; +case XFS_ATTRI_OP_FLAGS_REMOVE : +/* Log item, attr name */ +if (item->ri_total !=2 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +break ; +default : +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +/* Validate the attr name */ if (item->ri_buf[1 ].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_name_len)) {
Re: [PATCH 04/12] xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Check that the number of recovered log iovecs is what is expected for the xattri opcode is expecting.
Looks good:
PATCH 5/12 xfs: fix missing check for invalid attr flags
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The xattr scrubber doesn't check for undefined flags in shortform attr entries. Therefore, define a mask XFS_ATTR_ONDISK_MASK that has all possible XFS_ATTR_* flags in it, and use that to check for unknown bits in xchk_xattr_actor.
Refactor the check in the dabtree scanner function to use the new mask as well. The redundant checks need to be in place because the dabtree check examines the hash mappings and therefore needs to decode the attr leaf entries to compute the namehash. This happens before the walk of the xattr entries themselves.
--- fs/xfs/libxfs/xfs_da_format.h | 5 +++++ fs/xfs/scrub/attr.c | 13 +++++++++---- 2 files changed, 14 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +5 -0
@@ -719,8 +719,13 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) +#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_ONDISK_MASK (XFS_ATTR_NSP_ONDISK_MASK | \ + XFS_ATTR_LOCAL | \ + XFS_ATTR_INCOMPLETE) +#define XFS_ATTR_NAMESPACE_STR \ { XFS_ATTR_LOCAL, "local" }, \ { XFS_ATTR_ROOT, "root" }, \
fs/xfs/scrub/attr.c +9 -4
@@ -192,6 +192,11 @@ xchk_xattr_actor(if (xchk_should_terminate (sc, &error))return error; +if (attr_flags & ~XFS_ATTR_ONDISK_MASK ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno); +return -ECANCELED ; + } +if (attr_flags &XFS_ATTR_INCOMPLETE ) {/* Incomplete attr key, just mark the inode for preening. */ xchk_ino_set_preen (sc, ip->i_ino);
@@ -481,7 +486,6 @@ xchk_xattr_rec(xfs_dahash_t hash;int nameidx;int hdrsize; -unsigned int badflags;int error;ASSERT (blk->magic ==XFS_ATTR_LEAF_MAGIC );
@@ -511,10 +515,11 @@ xchk_xattr_rec(/* Retrieve the entry and check it. */ hash =be32_to_cpu (ent->hashval); - badflags = ~(XFS_ATTR_LOCAL |XFS_ATTR_ROOT |XFS_ATTR_SECURE | -XFS_ATTR_INCOMPLETE ); -if ((ent->flags & badflags) !=0 ) +if (ent->flags & ~XFS_ATTR_ONDISK_MASK ) {xchk_da_set_corrupt (ds, level); +return 0 ; + } +if (ent->flags &XFS_ATTR_LOCAL ) { lentry = (struct xfs_attr_leaf_name_local *) (((char *)bp->b_addr) + nameidx);
Re: [PATCH 05/12] xfs: fix missing check for invalid attr flags
fs/xfs/libxfs/xfs_da_format.h struct xfs_attr3_leafblock { · line 727
Quoted 3 lines · Darrick J. Wong
fs/xfs/libxfs/xfs_da_format.h +3
-0+#define XFS_ATTR_ONDISK_MASK (XFS_ATTR_NSP_ONDISK_MASK | \ + XFS_ATTR_LOCAL | \ + XFS_ATTR_INCOMPLETE)
Note that XFS_ATTR_LOCAL and XFS_ATTR_INCOMPLETE are not valid for short form directories. Should we check for that somewhere as well?
Otherwise looks good:
Re: [PATCH 05/12] xfs: fix missing check for invalid attr flags
Quoted 7 lines · Christoph Hellwig
fs/xfs/libxfs/xfs_da_format.h struct xfs_attr3_leafblock { · line 727
Quoted 3 lines · Darrick J. Wong
fs/xfs/libxfs/xfs_da_format.h +3
-0+#define XFS_ATTR_ONDISK_MASK (XFS_ATTR_NSP_ONDISK_MASK | \ + XFS_ATTR_LOCAL | \ + XFS_ATTR_INCOMPLETE)Note that
XFS_ATTR_LOCALandXFS_ATTR_INCOMPLETEare not valid for short form directories. Should we check for that somewhere as well?
Good point, xchk_xattr_check_sf should be flagging those too:
/*
* Shortform entries do not set LOCAL or INCOMPLETE, so the only
* valid flag bits here are for namespaces.
*/
if (sfe->flags & ~XFS_ATTR_NSP_ONDISK_MASK) {
xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
break;
}I'll tack that on the end.
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
PATCH 6/12 xfs: restructure xfs_attr_complete_op a bit
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Eliminate the local variable from this function so that we can streamline things a bit later when we add the PPTR_REPLACE op code.
--- fs/xfs/libxfs/xfs_attr.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_attr.c +4 -5
@@ -432,14 +432,13 @@ xfs_attr_complete_op( enum xfs_delattr_state replace_state) {struct xfs_da_args *args = attr->xattri_da_args; -bool do_replace = args->op_flags &XFS_DA_OP_REPLACE ; + +if (!(args->op_flags &XFS_DA_OP_REPLACE )) + replace_state =XFS_DAS_DONE ; args->op_flags &= ~XFS_DA_OP_REPLACE ; args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; -if (do_replace) -return replace_state; - -return XFS_DAS_DONE ; +return replace_state; } static int
Re: [PATCH 06/12] xfs: restructure xfs_attr_complete_op a bit
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Eliminate the local variable from this function so that we can streamline things a bit later when we add the
PPTR_REPLACEop code.
Looks good:
PATCH 7/12 xfs: use helpers to extract xattr op from opflags
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create helper functions to extract the xattr op from the ondisk xattri log item and the incore attr intent item. These will get more use in the patches that follow.
--- fs/xfs/libxfs/xfs_attr.h | 5 +++++ fs/xfs/xfs_attr_item.c | 16 ++++++++++------ 2 files changed, 15 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_attr.h +5 -0
@@ -529,6 +529,11 @@ struct xfs_attr_intent {struct xfs_bmbt_irec xattri_map; }; +static inline unsigned int +xfs_attr_intent_op (const struct xfs_attr_intent *attr) +{ +return attr->xattri_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +}/*======================================================================== * Function prototypes for the kernel.
fs/xfs/xfs_attr_item.c +10 -6
@@ -308,6 +308,12 @@ xfs_attrd_item_intent(return &ATTRD_ITEM (lip)->attrd_attrip->attri_item; } +static inline unsigned int +xfs_attr_log_item_op (const struct xfs_attri_log_format *attrp) +{ +return attrp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +} +/* Log an attr to the intent item. */ STATIC void xfs_attr_log_item(
@@ -466,8 +472,7 @@ xfs_attri_validate(struct xfs_mount *mp, struct xfs_attri_log_format *attrp) { - unsigned int op = attrp->alfi_op_flags & -XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + unsigned int op = xfs_attr_log_item_op(attrp);if (!xfs_is_using_logged_xattrs (mp))return false;
@@ -551,8 +556,7 @@ xfs_attri_recover_work( args = (struct xfs_da_args *)(attr +1 ); attr->xattri_da_args = args; - attr->xattri_op_flags = attrp->alfi_op_flags & -XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + attr->xattri_op_flags =xfs_attr_log_item_op (attrp);/* * We're reconstructing the deferred work state structure from the
@@ -573,7 +577,7 @@ xfs_attri_recover_work(XFS_DA_OP_LOGGED ; args->owner = args->dp->i_ino; -switch (attr->xattri_op_flags) { +switch (xfs_attr_intent_op (attr)) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : args->value = nv->value.i_addr;
@@ -757,7 +761,7 @@ xlog_recover_attri_commit_pass2( }/* Check the number of log iovecs makes sense for the op code. */ - op = attri_formatp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + op =xfs_attr_log_item_op (attri_formatp);switch (op) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :
Re: [PATCH 07/12] xfs: use helpers to extract xattr op from opflags
Looks good:
PATCH 8/12 xfs: validate recovered name buffers when recovering xattr items
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Strengthen the xattri log item recovery code by checking that we actually have the required name and newname buffers for whatever operation we're replaying.
--- fs/xfs/xfs_attr_item.c | 58 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 47 insertions(+), 11 deletions(-)
fs/xfs/xfs_attr_item.c +47 -11
@@ -741,22 +741,20 @@ xlog_recover_attri_commit_pass2(const void *attr_value =NULL ;const void *attr_name;size_t len; -unsigned int op; - - attri_formatp = item->ri_buf[0 ].i_addr; - attr_name = item->ri_buf[1 ].i_addr; +unsigned int op, i =0 ;/* Validate xfs_attri_log_format before the large memory allocation */ len =sizeof (struct xfs_attri_log_format ); -if (item->ri_buf[0 ].i_len != len) { +if (item->ri_buf[i].i_len != len) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len);return -EFSCORRUPTED ; } + attri_formatp = item->ri_buf[i].i_addr;if (!xfs_attri_validate (mp, attri_formatp)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len); + attri_formatp, len);return -EFSCORRUPTED ; }
@@ -785,31 +783,69 @@ xlog_recover_attri_commit_pass2( attri_formatp, len);return -EFSCORRUPTED ; } + i++;/* Validate the attr name */ -if (item->ri_buf[1 ].i_len != +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len); + attri_formatp, len);return -EFSCORRUPTED ; } + attr_name = item->ri_buf[i].i_addr;if (!xfs_attr_namecheck (attr_name, attri_formatp->alfi_name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[1 ].i_addr, item->ri_buf[1 ].i_len); + attri_formatp, len);return -EFSCORRUPTED ; } + i++;/* Validate the attr value, if present */ if (attri_formatp->alfi_value_len !=0 ) { -if (item->ri_buf[2 ].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_value_len)) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_value_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len);return -EFSCORRUPTED ; } - attr_value = item->ri_buf[2 ].i_addr; + attr_value = item->ri_buf[i].i_addr; + i++; + } + +/* +* Make sure we got the correct number of buffers for the operation +* that we just loaded. +*/ +if (i != item->ri_total) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } + +switch (op) { +case XFS_ATTRI_OP_FLAGS_REMOVE : +/* Regular remove operations operate only on names. */ +if (attr_value !=NULL || attri_formatp->alfi_value_len !=0 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } + fallthrough; +case XFS_ATTRI_OP_FLAGS_SET : +case XFS_ATTRI_OP_FLAGS_REPLACE : +/* +* Regular xattr set/remove/replace operations require a name +* and do not take a newname. Values are optional for set and +* replace. +*/ +if (attr_name ==NULL || attri_formatp->alfi_name_len ==0 ) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attri_formatp, len); +return -EFSCORRUPTED ; + } +break ; }/*
Re: [PATCH 08/12] xfs: validate recovered name buffers when recovering xattr items
Looks good:
PATCH 9/12 xfs: always set args->value in xfs_attri_item_recover
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Always set args->value to the recovered value buffer. This reduces the amount of code in the switch statement, and hence the amount of thinking that I have to do. We validated the recovered buffers, supposedly.
--- fs/xfs/xfs_attr_item.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/xfs/xfs_attr_item.c +2 -2
@@ -572,6 +572,8 @@ xfs_attri_recover_work( args->name = nv->name.i_addr; args->namelen = nv->name.i_len; args->hashval =xfs_da_hashname (args->name, args->namelen); + args->value = nv->value.i_addr; + args->valuelen = nv->value.i_len; args->attr_filter = attrp->alfi_attr_filter &XFS_ATTRI_FILTER_MASK ; args->op_flags =XFS_DA_OP_RECOVERY |XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ;
@@ -580,8 +582,6 @@ xfs_attri_recover_work(switch (xfs_attr_intent_op (attr)) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : - args->value = nv->value.i_addr; - args->valuelen = nv->value.i_len; args->total =xfs_attr_calc_size (args, &local);if (xfs_inode_hasattr (args->dp)) attr->xattri_dela_state =xfs_attr_init_replace_state (args);
Re: [PATCH 09/12] xfs: always set args->value in xfs_attri_item_recover
Looks good:
PATCH 10/12 xfs: use local variables for name and value length in _attri_commit_pass2
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
We're about to start using tagged unions in the xattr log format, so create a bunch of local variables in the recovery function so we only have to decode the log item fields once.
--- fs/xfs/xfs_attr_item.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
fs/xfs/xfs_attr_item.c +14 -11
@@ -738,9 +738,11 @@ xlog_recover_attri_commit_pass2(struct xfs_attri_log_item *attrip;struct xfs_attri_log_format *attri_formatp;struct xfs_attri_log_nameval *nv; -const void *attr_value =NULL ;const void *attr_name; +const void *attr_value =NULL ;size_t len; +unsigned int name_len =0 ; +unsigned int value_len =0 ;unsigned int op, i =0 ;/* Validate xfs_attri_log_format before the large memory allocation */
@@ -769,6 +771,8 @@ xlog_recover_attri_commit_pass2( attri_formatp, len);return -EFSCORRUPTED ; } + name_len = attri_formatp->alfi_name_len; + value_len = attri_formatp->alfi_value_len;break ;case XFS_ATTRI_OP_FLAGS_REMOVE :/* Log item, attr name */
@@ -777,6 +781,7 @@ xlog_recover_attri_commit_pass2( attri_formatp, len);return -EFSCORRUPTED ; } + name_len = attri_formatp->alfi_name_len;break ;default : XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
@@ -786,15 +791,14 @@ xlog_recover_attri_commit_pass2( i++;/* Validate the attr name */ -if (item->ri_buf[i].i_len != -xlog_calc_iovec_len (attri_formatp->alfi_name_len)) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ; } attr_name = item->ri_buf[i].i_addr; -if (!xfs_attr_namecheck (attr_name, attri_formatp->alfi_name_len)) { +if (!xfs_attr_namecheck (attr_name, name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -802,8 +806,8 @@ xlog_recover_attri_commit_pass2( i++;/* Validate the attr value, if present */ -if (attri_formatp->alfi_value_len !=0 ) { -if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (attri_formatp->alfi_value_len)) { +if (value_len !=0 ) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (value_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[0 ].i_addr, item->ri_buf[0 ].i_len);
@@ -827,7 +831,7 @@ xlog_recover_attri_commit_pass2(switch (op) {case XFS_ATTRI_OP_FLAGS_REMOVE :/* Regular remove operations operate only on names. */ -if (attr_value !=NULL || attri_formatp->alfi_value_len !=0 ) { +if (attr_value !=NULL || value_len !=0 ) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -840,7 +844,7 @@ xlog_recover_attri_commit_pass2(* and do not take a newname. Values are optional for set and * replace. */ -if (attr_name ==NULL || attri_formatp->alfi_name_len ==0 ) { +if (attr_name ==NULL || name_len ==0 ) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -853,9 +857,8 @@ xlog_recover_attri_commit_pass2(* name/value buffer to the recovered incore log item and drop our * reference. */ - nv =xfs_attri_log_nameval_alloc (attr_name, - attri_formatp->alfi_name_len, attr_value, - attri_formatp->alfi_value_len); + nv =xfs_attri_log_nameval_alloc (attr_name, name_len, + attr_value, value_len); attrip =xfs_attri_init (mp, nv);memcpy (&attrip->attri_format, attri_formatp, len);
Re: [PATCH 10/12] xfs: use local variables for name and value length in _attri_commit_pass2
Looks good:
PATCH 11/12 xfs: refactor name/length checks in xfs_attri_validate
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Move the name and length checks into the attr op switch statement so that we can perform more specific checks of the value length. Over the next few patches we're going to add new attr op flags with different validation requirements.
While we're at it, remove the incorrect comment.
--- fs/xfs/xfs_attr_item.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
fs/xfs/xfs_attr_item.c +11 -8
@@ -486,23 +486,26 @@ xfs_attri_validate(if (attrp->alfi_attr_filter & ~XFS_ATTRI_FILTER_MASK )return false; -/* alfi_op_flags should be either a set or remove */ switch (op) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : +if (attrp->alfi_value_len >XATTR_SIZE_MAX ) +return false; +if (attrp->alfi_name_len ==0 || + attrp->alfi_name_len >XATTR_NAME_MAX ) +return false; +break ;case XFS_ATTRI_OP_FLAGS_REMOVE : +if (attrp->alfi_value_len !=0 ) +return false; +if (attrp->alfi_name_len ==0 || + attrp->alfi_name_len >XATTR_NAME_MAX ) +return false;break ;default :return false; } -if (attrp->alfi_value_len >XATTR_SIZE_MAX ) -return false; - -if ((attrp->alfi_name_len >XATTR_NAME_MAX ) || - (attrp->alfi_name_len ==0 )) -return false; -return xfs_verify_ino (mp, attrp->alfi_ino); }
Re: [PATCH 11/12] xfs: refactor name/length checks in xfs_attri_validate
Looks good:
PATCH 12/12 xfs: enforce one namespace per attribute
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a standardized helper function to enforce one namespace bit per extended attribute, and refactor all the open-coded hweight logic. This function is not a static inline to avoid porting hassles in userspace.
--- fs/xfs/libxfs/xfs_attr.c | 15 +++++++++++++++ fs/xfs/libxfs/xfs_attr.h | 4 +++- fs/xfs/libxfs/xfs_attr_leaf.c | 7 ++++++- fs/xfs/scrub/attr.c | 12 +++++------- fs/xfs/scrub/attr_repair.c | 4 +--- fs/xfs/xfs_attr_item.c | 10 ++++++++-- fs/xfs/xfs_attr_list.c | 11 +++++++---- 7 files changed, 45 insertions(+), 18 deletions(-)
fs/xfs/libxfs/xfs_attr.c +15 -0
@@ -275,6 +275,8 @@ xfs_attr_get(if (xfs_is_shutdown (args->dp->i_mount))return -EIO ; +if (!xfs_attr_namecheck (args->attr_filter, args->name, args->namelen)) +return -EFSCORRUPTED ;if (!args->owner) args->owner = args->dp->i_ino;
@@ -950,6 +952,8 @@ xfs_attr_set(if (xfs_is_shutdown (dp->i_mount))return -EIO ; +if (!xfs_attr_namecheck (args->attr_filter, args->name, args->namelen)) +return -EFSCORRUPTED ; error =xfs_qm_dqattach (dp);if (error)
@@ -1530,12 +1534,23 @@ xfs_attr_node_get(return error; } +/* Enforce that there is at most one namespace bit per attr. */ +inline bool xfs_attr_check_namespace (unsigned int attr_flags) +{ +return hweight32 (attr_flags &XFS_ATTR_NSP_ONDISK_MASK ) <2 ; +} +/* Returns true if the attribute entry name is valid. */ bool xfs_attr_namecheck ( +unsigned int attr_flags,const void *name,size_t length) { +/* Only one namespace bit allowed. */ +if (!xfs_attr_check_namespace (attr_flags)) +return false; +/* * MAXNAMELEN includes the trailing null, but (name/length) leave it * out, so use >= for the length check.
fs/xfs/libxfs/xfs_attr.h +3 -1
@@ -552,7 +552,9 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_check_namespace (unsigned int attr_flags); +bool xfs_attr_namecheck (unsigned int attr_flags,const void *name, +size_t length);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);
fs/xfs/libxfs/xfs_attr_leaf.c +6 -1
@@ -949,6 +949,11 @@ xfs_attr_shortform_to_leaf( nargs.hashval =xfs_da_hashname (sfe->nameval, sfe->namelen); nargs.attr_filter = sfe->flags &XFS_ATTR_NSP_ONDISK_MASK ; +if (!xfs_attr_check_namespace (sfe->flags)) { +xfs_da_mark_sick (args); + error = -EFSCORRUPTED ; + goto out; + } error =xfs_attr3_leaf_lookup_int (bp, &nargs);/* set a->index */ ASSERT (error == -ENOATTR ); error =xfs_attr3_leaf_add (bp, &nargs);
@@ -1062,7 +1067,7 @@ xfs_attr_shortform_verify(* one namespace flag per xattr, so we can just count the * bits (i.e. hweight) here. */ -if (hweight8 (sfep->flags &XFS_ATTR_NSP_ONDISK_MASK ) >1 ) +if (!xfs_attr_check_namespace (sfep->flags))return __this_address; sfep = next_sfep;
fs/xfs/scrub/attr.c +5 -7
@@ -203,14 +203,8 @@ xchk_xattr_actor(return 0 ; } -/* Only one namespace bit allowed. */ -if (hweight32 (attr_flags &XFS_ATTR_NSP_ONDISK_MASK ) >1 ) { -xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno); -return -ECANCELED ; - } -/* Does this name make sense? */ -if (!xfs_attr_namecheck (name, namelen)) { +if (!xfs_attr_namecheck (attr_flags, name, namelen)) {xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno);return -ECANCELED ; }
@@ -519,6 +513,10 @@ xchk_xattr_rec(xchk_da_set_corrupt (ds, level);return 0 ; } +if (!xfs_attr_check_namespace (ent->flags)) { +xchk_da_set_corrupt (ds, level); +return 0 ; + }if (ent->flags &XFS_ATTR_LOCAL ) { lentry = (struct xfs_attr_leaf_name_local *)
fs/xfs/scrub/attr_repair.c +1 -3
@@ -123,12 +123,10 @@ xrep_xattr_want_salvage(return false;if (namelen >XATTR_NAME_MAX || namelen <=0 )return false; -if (!xfs_attr_namecheck (name, namelen)) +if (!xfs_attr_namecheck (attr_flags, name, namelen))return false;if (valuelen >XATTR_SIZE_MAX || valuelen <0 )return false; -if (hweight32 (attr_flags &XFS_ATTR_NSP_ONDISK_MASK ) >1 ) -return false;return true; }
fs/xfs/xfs_attr_item.c +8 -2
@@ -486,6 +486,10 @@ xfs_attri_validate(if (attrp->alfi_attr_filter & ~XFS_ATTRI_FILTER_MASK )return false; +if (!xfs_attr_check_namespace (attrp->alfi_attr_filter & +XFS_ATTR_NSP_ONDISK_MASK )) +return false; +switch (op) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :
@@ -629,7 +633,8 @@ xfs_attr_recover_work(*/ attrp = &attrip->attri_format;if (!xfs_attri_validate (mp, attrp) || - !xfs_attr_namecheck (nv->name.i_addr, nv->name.i_len)) + !xfs_attr_namecheck (attrp->alfi_attr_filter, nv->name.i_addr, + nv->name.i_len))return -EFSCORRUPTED ; attr =xfs_attri_recover_work (mp, dfp, attrp, &ip, nv);
@@ -801,7 +806,8 @@ xlog_recover_attri_commit_pass2( } attr_name = item->ri_buf[i].i_addr; -if (!xfs_attr_namecheck (attr_name, name_len)) { +if (!xfs_attr_namecheck (attri_formatp->alfi_attr_filter, attr_name, + name_len)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
fs/xfs/xfs_attr_list.c +7 -4
@@ -82,7 +82,8 @@ xfs_attr_shortform_list( (dp->i_af.if_bytes + sf->count * 16) < context->bufsize)) {for (i =0 , sfe =xfs_attr_sf_firstentry (sf); i < sf->count; i++) {if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sfe->nameval, + !xfs_attr_namecheck (sfe->flags, + sfe->nameval, sfe->namelen))) {xfs_dirattr_mark_sick (context->dp,XFS_ATTR_FORK );return -EFSCORRUPTED ;
@@ -122,7 +123,8 @@ xfs_attr_shortform_list(for (i =0 , sfe =xfs_attr_sf_firstentry (sf); i < sf->count; i++) {if (unlikely ( ((char *)sfe < (char *)sf) || - ((char *)sfe >= ((char *)sf + dp->i_af.if_bytes)))) { + ((char *)sfe >= ((char *)sf + dp->i_af.if_bytes)) || + !xfs_attr_check_namespace (sfe->flags))) { XFS_CORRUPTION_ERROR("xfs_attr_shortform_list", XFS_ERRLEVEL_LOW, context->dp->i_mount, sfe,
@@ -177,7 +179,7 @@ xfs_attr_shortform_list( cursor->offset =0 ; }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sbp->name, + !xfs_attr_namecheck (sbp->flags, sbp->name, sbp->namelen))) {xfs_dirattr_mark_sick (context->dp,XFS_ATTR_FORK ); error = -EFSCORRUPTED ;
@@ -502,7 +504,8 @@ xfs_attr3_leaf_list_int( }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (name, namelen))) { + !xfs_attr_namecheck (entry->flags, name, + namelen))) {xfs_dirattr_mark_sick (context->dp,XFS_ATTR_FORK );return -EFSCORRUPTED ; }
Re: [PATCH 12/12] xfs: enforce one namespace per attribute
Looks good:
PATCH 5/9 xfs: Parent Pointers
In reply to Darrick J. WongHi all,
This is the latest parent pointer attributes for xfs. The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point.
Directory parent pointers are stored as namespaced extended attributes of a file. Because parent pointers are an indivisible tuple of (dirent_name, parent_ino, parent_gen) we cannot use the usual attr name lookup functions to find a parent pointer. This is solvable by introducing a new lookup mode that checks both the name and the value of the xattr.
Therefore, introduce this new name-value lookup mode that's gated on the XFS_ATTR_PARENT namespace. This requires the introduction of new opcodes for the extended attribute update log intent items, which actually means that parent pointers (itself an INCOMPAT feature) does not depend on the LOGGED_XATTRS log incompat feature bit.
To reduce collisions on the dirent names of parent pointers, introduce a new attr hash mode that is the dir2 namehash of the dirent name xor'd with the parent inode number.
At this point, Allison has moved on to other things, so I've merged her patchset into djwong-dev for merging.
Updates since v12 [djwong]:
Rebase on 6.9-rc and update the online fsck design document. Redesign the ondisk format to use the name-value lookups to get us back to the point where the attr is (dirent_name -> parent_ino/gen).
Updates since v11 [djwong]:
Rebase on 6.4-rc and make some tweaks and bugfixes to enable the repair prototypes. Merge with djwong-dev and make online repair actually work.
Updates since v10 [djwong]:
Merge in the ondisk format changes to get rid of the diroffset conflicts with the parent pointer repair code, rebase the entire series with the attr vlookup changes first, and merge all the other random fixes.
Updates since v9:
Reordered patches 2 and 3 to be 6 and 7
xfs: Add xfs_verify_pptr
moved parent pointer validators to xfs_parent
xfs: Add parent pointer ioctl
Extra validation checks for fs id
added missing release for the inode
use GFP_KERNEL flags for malloc/realloc reworked ioctl to use pptr listenty and flex array
NEW
xfs: don't remove the attr fork when parent pointers are enabled
NEW
directory lookups should return diroffsets too
NEW
xfs: move/add parent pointer validators to xfs_parent
Updates since v8:
xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec
xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas
xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates
xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free
xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once Fix xfs_rename to disallow reservationless quotas Fix double unlock on dqattach fail
Move parent frees to out_release_wip
xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename
Questions comments and feedback appreciated!
Thanks all!
Allison
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
xfsdocs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-documentation.git/log/?h=pptrs
---
Commits in this patchset:
* xfs: rearrange xfs_attr_match parameters
* xfs: check the flags earlier in xfs_attr_match
* xfs: move xfs_attr_defer_add to xfs_attr_item.c
* xfs: create a separate hashname function for extended attributes
* xfs: add parent pointer support to attribute code
* xfs: define parent pointer ondisk extended attribute format
* xfs: allow xattr matching on name and value for local/sf attrs
* xfs: allow logged xattr operations if parent pointers are enabled
* xfs: log parent pointer xattr removal operations
* xfs: log parent pointer xattr setting operations
* xfs: log parent pointer xattr replace operations
* xfs: record inode generation in xattr update log intent items
* xfs: Expose init_xattrs in xfs_create_tmpfile
* xfs: add parent pointer validator functions
* xfs: extend transaction reservations for parent attributes
* xfs: create a hashname function for parent pointers
* xfs: parent pointer attribute creation
* xfs: add parent attributes to link
* xfs: add parent attributes to symlink
* xfs: remove parent pointers in unlink
* xfs: Add parent pointers to rename
* xfs: Add parent pointers to xfs_cross_rename
* xfs: Filter XFS_ATTR_PARENT for getfattr
* xfs: pass the attr value to put_listent when possible
* xfs: move handle ioctl code to xfs_handle.c
* xfs: split out handle management helpers a bit
* xfs: Add parent pointer ioctls
* xfs: don't remove the attr fork when parent pointers are enabled
* xfs: Add the parent pointer support to the superblock version 5.
* xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
* xfs: drop compatibility minimum log size computations for reflink
* xfs: enable parent pointers
--- fs/xfs/Makefile | 3 fs/xfs/libxfs/xfs_attr.c | 92 ++-- fs/xfs/libxfs/xfs_attr.h | 23 + fs/xfs/libxfs/xfs_attr_leaf.c | 81 +++ fs/xfs/libxfs/xfs_attr_sf.h | 1 fs/xfs/libxfs/xfs_da_btree.h | 4 fs/xfs/libxfs/xfs_da_format.h | 25 + fs/xfs/libxfs/xfs_format.h | 4 fs/xfs/libxfs/xfs_fs.h | 79 +++ fs/xfs/libxfs/xfs_log_format.h | 25 + fs/xfs/libxfs/xfs_log_rlimit.c | 46 ++ fs/xfs/libxfs/xfs_ondisk.h | 6 fs/xfs/libxfs/xfs_parent.c | 296 +++++++++++++ fs/xfs/libxfs/xfs_parent.h | 99 ++++ fs/xfs/libxfs/xfs_sb.c | 4 fs/xfs/libxfs/xfs_trans_resv.c | 326 ++++++++++++-- fs/xfs/libxfs/xfs_trans_space.c | 121 +++++ fs/xfs/libxfs/xfs_trans_space.h | 25 + fs/xfs/scrub/attr.c | 15 + fs/xfs/scrub/dir_repair.c | 2 fs/xfs/scrub/orphanage.c | 5 fs/xfs/scrub/parent_repair.c | 3 fs/xfs/scrub/symlink_repair.c | 2 fs/xfs/scrub/tempfile.c | 2 fs/xfs/xfs_attr_item.c | 320 +++++++++++++- fs/xfs/xfs_attr_item.h | 12 + fs/xfs/xfs_attr_list.c | 13 - fs/xfs/xfs_handle.c | 906 +++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_handle.h | 33 + fs/xfs/xfs_inode.c | 218 ++++++++- fs/xfs/xfs_inode.h | 2 fs/xfs/xfs_ioctl.c | 594 -------------------------- fs/xfs/xfs_ioctl.h | 28 - fs/xfs/xfs_ioctl32.c | 1 fs/xfs/xfs_iops.c | 15 + fs/xfs/xfs_super.c | 14 + fs/xfs/xfs_symlink.c | 30 + fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 95 ++++ fs/xfs/xfs_xattr.c | 13 + fs/xfs/xfs_xattr.h | 2 41 files changed, 2763 insertions(+), 823 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/libxfs/xfs_trans_space.c create mode 100644 fs/xfs/xfs_handle.c create mode 100644 fs/xfs/xfs_handle.h
PATCH 1/32 xfs: rearrange xfs_attr_match parameters
From: Darrick J. Wong <[email protected]>
Rearrange the parameters to this function so that they match the order of attr listent: attr_flags -> name -> namelen -> value -> valuelen.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +12 -11
@@ -509,9 +509,9 @@ xfs_attr3_leaf_read(static bool xfs_attr_match (struct xfs_da_args *args, -uint8_t namelen, -unsigned char *name, -int flags) +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen) {if (args->namelen != namelen)
@@ -521,12 +521,12 @@ xfs_attr_match(/* Recovery ignores the INCOMPLETE flag. */ if ((args->op_flags &XFS_DA_OP_RECOVERY ) && - args->attr_filter == (flags &XFS_ATTR_NSP_ONDISK_MASK )) + args->attr_filter == (attr_flags &XFS_ATTR_NSP_ONDISK_MASK ))return true;/* All remaining matches need to be filtered by INCOMPLETE state. */ if (args->attr_filter != - (flags & (XFS_ATTR_NSP_ONDISK_MASK |XFS_ATTR_INCOMPLETE ))) + (attr_flags & (XFS_ATTR_NSP_ONDISK_MASK |XFS_ATTR_INCOMPLETE )))return false;return true; }
@@ -745,8 +745,8 @@ xfs_attr_sf_findname(for (sfe =xfs_attr_sf_firstentry (sf); sfe <xfs_attr_sf_endptr (sf); sfe =xfs_attr_sf_nextentry (sfe)) { -if (xfs_attr_match (args, sfe->namelen, sfe->nameval, - sfe->flags)) +if (xfs_attr_match (args, sfe->flags, sfe->nameval, + sfe->namelen))return sfe; }
@@ -2442,15 +2442,16 @@ xfs_attr3_leaf_lookup_int(*/ if (entry->flags &XFS_ATTR_LOCAL ) { name_loc =xfs_attr3_leaf_name_local (leaf, probe); -if (!xfs_attr_match (args, name_loc->namelen, - name_loc->nameval, entry->flags)) +if (!xfs_attr_match (args, entry->flags, + name_loc->nameval, + name_loc->namelen))continue ; args->index = probe;return -EEXIST ; }else { name_rmt =xfs_attr3_leaf_name_remote (leaf, probe); -if (!xfs_attr_match (args, name_rmt->namelen, - name_rmt->name, entry->flags)) +if (!xfs_attr_match (args, entry->flags, name_rmt->name, + name_rmt->namelen))continue ; args->index = probe; args->rmtvaluelen =be32_to_cpu (name_rmt->valuelen);
Re: [PATCH 01/32] xfs: rearrange xfs_attr_match parameters
Looks good:
PATCH 2/32 xfs: check the flags earlier in xfs_attr_match
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Checking the flags match is much cheaper than a memcmp, so do it early on in xfs_attr_match, and also add a little helper to calculate the match mask right under the comment explaining the logic for it.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +10 -9
@@ -506,6 +506,13 @@ xfs_attr3_leaf_read(* INCOMPLETE flag will not be set in attr->attr_filter, but rather * XFS_DA_OP_RECOVERY will be set in args->op_flags. */ +static inline unsigned int xfs_attr_match_mask (const struct xfs_da_args *args) +{ +if (args->op_flags &XFS_DA_OP_RECOVERY ) +return XFS_ATTR_NSP_ONDISK_MASK ; +return XFS_ATTR_NSP_ONDISK_MASK |XFS_ATTR_INCOMPLETE ; +} + static bool xfs_attr_match( struct xfs_da_args *args,
@@ -513,21 +520,15 @@ xfs_attr_match(const unsigned char *name, unsigned int namelen) { + unsigned int mask = xfs_attr_match_mask(args);if (args->namelen != namelen)return false; +if ((args->attr_filter & mask) != (attr_flags & mask)) +return false;if (memcmp (args->name, name, namelen) !=0 )return false; -/* Recovery ignores the INCOMPLETE flag. */ -if ((args->op_flags &XFS_DA_OP_RECOVERY ) && - args->attr_filter == (attr_flags &XFS_ATTR_NSP_ONDISK_MASK )) -return true; - -/* All remaining matches need to be filtered by INCOMPLETE state. */ -if (args->attr_filter != - (attr_flags & (XFS_ATTR_NSP_ONDISK_MASK |XFS_ATTR_INCOMPLETE ))) -return false;return true; }
PATCH 3/32 xfs: move xfs_attr_defer_add to xfs_attr_item.c
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Move the code that adds the incore xfs_attr_item deferred work data to a transaction live with the ATTRI log item code. This means that the upper level extended attribute code no longer has to know about the inner workings of the ATTRI log items.
--- fs/xfs/libxfs/xfs_attr.c | 37 +++---------------------------------- fs/xfs/xfs_attr_item.c | 30 ++++++++++++++++++++++++++++++ fs/xfs/xfs_attr_item.h | 8 ++++++++ 3 files changed, 41 insertions(+), 34 deletions(-)
fs/xfs/libxfs/xfs_attr.c +3 -34
@@ -903,37 +903,6 @@ xfs_attr_lookup(return error; } -static void -xfs_attr_defer_add ( -struct xfs_da_args *args, -unsigned int op_flags) -{ - -struct xfs_attr_intent *new; - - new =kmem_cache_zalloc (xfs_attr_intent_cache, -GFP_KERNEL | __GFP_NOFAIL); - new->xattri_op_flags = op_flags; - new->xattri_da_args = args; - -switch (op_flags) { -case XFS_ATTRI_OP_FLAGS_SET : - new->xattri_dela_state =xfs_attr_init_add_state (args); -break ; -case XFS_ATTRI_OP_FLAGS_REPLACE : - new->xattri_dela_state =xfs_attr_init_replace_state (args); -break ; -case XFS_ATTRI_OP_FLAGS_REMOVE : - new->xattri_dela_state =xfs_attr_init_remove_state (args); -break ; -default : -ASSERT (0 ); - } - -xfs_defer_add (args->trans, &new->xattri_list, &xfs_attr_defer_type); -trace_xfs_attr_defer_add (new->xattri_dela_state, args->dp); -} -/* * Note: If args->value is NULL the attribute will be removed, just like the * Linux ->setattr API.
@@ -1023,14 +992,14 @@ xfs_attr_set(case -EEXIST :if (!args->value) {/* if no value, we are performing a remove operation */ -xfs_attr_defer_add (args,XFS_ATTRI_OP_FLAGS_REMOVE ); +xfs_attr_defer_add (args,XFS_ATTR_DEFER_REMOVE );break ; }/* Pure create fails if the attr already exists */ if (args->xattr_flags &XATTR_CREATE ) goto out_trans_cancel; -xfs_attr_defer_add (args,XFS_ATTRI_OP_FLAGS_REPLACE ); +xfs_attr_defer_add (args,XFS_ATTR_DEFER_REPLACE );break ;case -ENOATTR :/* Can't remove what isn't there. */
@@ -1040,7 +1009,7 @@ xfs_attr_set(/* Pure replace fails if no existing attr to replace. */ if (args->xattr_flags &XATTR_REPLACE ) goto out_trans_cancel; -xfs_attr_defer_add (args,XFS_ATTRI_OP_FLAGS_SET ); +xfs_attr_defer_add (args,XFS_ATTR_DEFER_SET );break ;default : goto out_trans_cancel;
fs/xfs/xfs_attr_item.c +30 -0
@@ -723,6 +723,36 @@ xfs_attr_create_done(return &attrdp->attrd_item; } +void +xfs_attr_defer_add ( +struct xfs_da_args *args, +enum xfs_attr_defer_op op) +{ +struct xfs_attr_intent *new; + + new =kmem_cache_zalloc (xfs_attr_intent_cache, +GFP_NOFS | __GFP_NOFAIL); + new->xattri_da_args = args; + +switch (op) { +case XFS_ATTR_DEFER_SET : + new->xattri_op_flags =XFS_ATTRI_OP_FLAGS_SET ; + new->xattri_dela_state =xfs_attr_init_add_state (args); +break ; +case XFS_ATTR_DEFER_REPLACE : + new->xattri_op_flags =XFS_ATTRI_OP_FLAGS_REPLACE ; + new->xattri_dela_state =xfs_attr_init_replace_state (args); +break ; +case XFS_ATTR_DEFER_REMOVE : + new->xattri_op_flags =XFS_ATTRI_OP_FLAGS_REMOVE ; + new->xattri_dela_state =xfs_attr_init_remove_state (args); +break ; + } + +xfs_defer_add (args->trans, &new->xattri_list, &xfs_attr_defer_type); +trace_xfs_attr_defer_add (new->xattri_dela_state, args->dp); +} +const struct xfs_defer_op_type xfs_attr_defer_type = { .name ="attr" , .max_items =1 ,
fs/xfs/xfs_attr_item.h +8 -0
@@ -51,4 +51,12 @@ struct xfs_attrd_log_item {extern struct kmem_cache *xfs_attri_cache;extern struct kmem_cache *xfs_attrd_cache; +enum xfs_attr_defer_op { +XFS_ATTR_DEFER_SET , +XFS_ATTR_DEFER_REMOVE , +XFS_ATTR_DEFER_REPLACE , +}; + +void xfs_attr_defer_add (struct xfs_da_args *args,enum xfs_attr_defer_op op); +#endif /* __XFS_ATTR_ITEM_H__ */
Re: [PATCH 03/32] xfs: move xfs_attr_defer_add to xfs_attr_item.c
Looks good:
PATCH 4/32 xfs: create a separate hashname function for extended attributes
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a separate function to compute name hashvalues for extended attributes. When we get to parent pointers we'll be altering the rules so that metadump obfuscation doesn't turn heinous.
--- fs/xfs/libxfs/xfs_attr.c | 28 ++++++++++++++++++++++++++-- fs/xfs/libxfs/xfs_attr.h | 14 ++++++++++++++ fs/xfs/libxfs/xfs_attr_leaf.c | 3 +-- fs/xfs/scrub/attr.c | 11 ++++++++--- fs/xfs/xfs_attr_item.c | 2 +- fs/xfs/xfs_attr_list.c | 5 ++++- 6 files changed, 54 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_attr.c +26 -2
@@ -282,7 +282,7 @@ xfs_attr_get( args->owner = args->dp->i_ino; args->geo = args->dp->i_mount->m_attr_geo; args->whichfork =XFS_ATTR_FORK ; - args->hashval =xfs_da_hashname (args->name, args->namelen); +xfs_attr_sethash (args);/* Entirely possible to look up a name which doesn't exist */ args->op_flags =XFS_DA_OP_OKNOENT ;
@@ -417,6 +417,30 @@ xfs_attr_sf_addname(return error; } +/* Compute the hash value for a user/root/secure extended attribute */ +xfs_dahash_t +xfs_attr_hashname ( +const uint8_t *name, +int namelen) +{ +return xfs_da_hashname (name, namelen); +} + +/* Compute the hash value for any extended attribute from any namespace. */ +xfs_dahash_t +xfs_attr_hashval ( +struct xfs_mount *mp, +unsigned int attr_flags, +const uint8_t *name, +int namelen, +const void *value, +int valuelen) +{ +ASSERT (xfs_attr_check_namespace (attr_flags)); + +return xfs_attr_hashname (name, namelen); +} +/* * Handle the state change on completion of a multi-state attr operation. *
@@ -932,7 +956,7 @@ xfs_attr_set( args->owner = args->dp->i_ino; args->geo = mp->m_attr_geo; args->whichfork =XFS_ATTR_FORK ; - args->hashval =xfs_da_hashname (args->name, args->namelen); +xfs_attr_sethash (args);/* * We have no control over the attribute names that userspace passes us
fs/xfs/libxfs/xfs_attr.h +14 -0
@@ -620,6 +620,20 @@ xfs_attr_init_replace_state(struct xfs_da_args *args)return xfs_attr_init_add_state (args); } +xfs_dahash_t xfs_attr_hashname (const uint8_t *name,int namelen); + +xfs_dahash_t xfs_attr_hashval (struct xfs_mount *mp,unsigned int attr_flags, +const uint8_t *name,int namelen,const void *value, +int valuelen); + +/* Set the hash value for any extended attribute from any namespace. */ +static inline void xfs_attr_sethash (struct xfs_da_args *args) +{ + args->hashval =xfs_attr_hashval (args->dp->i_mount, args->attr_filter, + args->name, args->namelen, + args->value, args->valuelen); +} +extern struct kmem_cache *xfs_attr_intent_cache;int __init xfs_attr_intent_init_cache (void );void xfs_attr_intent_destroy_cache (void );
fs/xfs/libxfs/xfs_attr_leaf.c +1 -2
@@ -947,14 +947,13 @@ xfs_attr_shortform_to_leaf( nargs.namelen = sfe->namelen; nargs.value = &sfe->nameval[nargs.namelen]; nargs.valuelen = sfe->valuelen; - nargs.hashval =xfs_da_hashname (sfe->nameval, - sfe->namelen); nargs.attr_filter = sfe->flags &XFS_ATTR_NSP_ONDISK_MASK ;if (!xfs_attr_check_namespace (sfe->flags)) {xfs_da_mark_sick (args); error = -EFSCORRUPTED ; goto out; } +xfs_attr_sethash (&nargs); error =xfs_attr3_leaf_lookup_int (bp, &nargs);/* set a->index */ ASSERT (error == -ENOATTR ); error =xfs_attr3_leaf_add (bp, &nargs);
fs/xfs/scrub/attr.c +8 -3
@@ -179,7 +179,6 @@ xchk_xattr_actor( .dp = ip, .name = name, .namelen = namelen, - .hashval = xfs_da_hashname(name, namelen), .trans = sc->tp, .valuelen = valuelen, .owner = ip->i_ino,
@@ -230,6 +229,7 @@ xchk_xattr_actor( args.value = ab->value; +xfs_attr_sethash (&args); error =xfs_attr_get_ilocked (&args);/* ENODATA means the hash lookup failed and the attr is bad */ if (error == -ENODATA )
@@ -525,7 +525,10 @@ xchk_xattr_rec(xchk_da_set_corrupt (ds, level); goto out; } - calc_hash =xfs_da_hashname (lentry->nameval, lentry->namelen); + calc_hash =xfs_attr_hashval (mp, ent->flags, lentry->nameval, + lentry->namelen, + lentry->nameval + lentry->namelen, +be16_to_cpu (lentry->valuelen)); }else { rentry = (struct xfs_attr_leaf_name_remote *) (((char *)bp->b_addr) + nameidx);
@@ -533,7 +536,9 @@ xchk_xattr_rec(xchk_da_set_corrupt (ds, level); goto out; } - calc_hash =xfs_da_hashname (rentry->name, rentry->namelen); + calc_hash =xfs_attr_hashval (mp, ent->flags, rentry->name, + rentry->namelen,NULL , +be32_to_cpu (rentry->valuelen)); }if (calc_hash != hash)xchk_da_set_corrupt (ds, level);
fs/xfs/xfs_attr_item.c +1 -1
@@ -578,13 +578,13 @@ xfs_attri_recover_work( args->whichfork =XFS_ATTR_FORK ; args->name = nv->name.i_addr; args->namelen = nv->name.i_len; - args->hashval =xfs_da_hashname (args->name, args->namelen); args->value = nv->value.i_addr; args->valuelen = nv->value.i_len; args->attr_filter = attrp->alfi_attr_filter &XFS_ATTRI_FILTER_MASK ; args->op_flags =XFS_DA_OP_RECOVERY |XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ; args->owner = args->dp->i_ino; +xfs_attr_sethash (args);switch (xfs_attr_intent_op (attr)) {case XFS_ATTRI_OP_FLAGS_SET :
fs/xfs/xfs_attr_list.c +4 -1
@@ -135,12 +135,15 @@ xfs_attr_shortform_list( } sbp->entno = i; - sbp->hash =xfs_da_hashname (sfe->nameval, sfe->namelen); sbp->name = sfe->nameval; sbp->namelen = sfe->namelen;/* These are bytes, and both on-disk, don't endian-flip */ sbp->valuelen = sfe->valuelen; sbp->flags = sfe->flags; + sbp->hash =xfs_attr_hashval (dp->i_mount, sfe->flags, + sfe->nameval, sfe->namelen, + sfe->nameval + sfe->namelen, + sfe->valuelen); sfe =xfs_attr_sf_nextentry (sfe); sbp++; nsbuf++;
Re: [PATCH 04/32] xfs: create a separate hashname function for extended attributes
Looks good:
PATCH 5/32 xfs: add parent pointer support to attribute code
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer entries; it uses reserved blocks like XFS_ATTR_ROOT.
--- fs/xfs/libxfs/xfs_da_format.h | 9 +++++++-- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_trace.h | 3 ++- 3 files changed, 10 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +7 -2
@@ -714,13 +714,17 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */ #define XFS_ATTR_SECURE_BIT 2/* limit access to secure attrs */ +#define XFS_ATTR_PARENT_BIT 3/* parent pointer attrs */ #define XFS_ATTR_INCOMPLETE_BIT 7/* attr in middle of create/delete */ #define XFS_ATTR_LOCAL (1u << XFS_ATTR_LOCAL_BIT)#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT) +#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | \ + XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT)#define XFS_ATTR_ONDISK_MASK (XFS_ATTR_NSP_ONDISK_MASK | \ XFS_ATTR_LOCAL | \
@@ -729,7 +733,8 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_NAMESPACE_STR \ { XFS_ATTR_LOCAL, "local" }, \ { XFS_ATTR_ROOT, "root" }, \ - { XFS_ATTR_SECURE, "secure" } + { XFS_ATTR_SECURE, "secure" }, \ + { XFS_ATTR_PARENT, "parent" }/* * Alignment for namelist and valuelist entries (since they are mixed
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -1034,6 +1034,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/*
fs/xfs/xfs_trace.h +2 -1
@@ -91,7 +91,8 @@ struct xfs_exchrange;#define XFS_ATTR_FILTER_FLAGS \ { XFS_ATTR_ROOT, "ROOT" }, \ { XFS_ATTR_SECURE, "SECURE" }, \ - { XFS_ATTR_INCOMPLETE, "INCOMPLETE" } + { XFS_ATTR_INCOMPLETE, "INCOMPLETE" }, \ + { XFS_ATTR_PARENT, "PARENT" } DECLARE_EVENT_CLASS(xfs_attr_list_class, TP_PROTO(struct xfs_attr_list_context *ctx),
Re: [PATCH 05/32] xfs: add parent pointer support to attribute code
Looks good:
PATCH 6/32 xfs: define parent pointer ondisk extended attribute format
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to define the parent pointer attribute format before we start adding support for it into all the code that needs to use it. The EA format we will use encodes the following information:
name={dirent name}
value={parent inumber, parent inode generation}
hash=xfs_dir2_hashname(dirent name) ^ (parent_inumber)The inode/gen gives all the information we need to reliably identify the parent without requiring child->parent lock ordering, and allows userspace to do pathname component level reconstruction without the kernel ever needing to verify the parent itself as part of ioctl calls.
By using the name-value lookup mode in the extended attribute code to match parent pointers using both the xattr name and value, we can identify the exact parent pointer EA we need to modify/remove in rename/unlink operations without searching the entire EA space.
By storing the dirent name, we have enough information to be able to validate and reconstruct damaged directory trees. Earlier iterations of this patchset encoded the directory offset in the parent pointer key, but this format required repair to keep that in sync across directory rebuilds, which is unnecessary complexity.
--- fs/xfs/libxfs/xfs_da_format.h | 13 +++++++++++++ fs/xfs/libxfs/xfs_ondisk.h | 1 + 2 files changed, 14 insertions(+)
fs/xfs/libxfs/xfs_da_format.h +13 -0
@@ -890,4 +890,17 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); +/* +* Parent pointer attribute format definition +* +* The xattr name contains the dirent name. +* The xattr value encodes the parent inode number and generation to ease +* opening parents by handle. +* The xattr hashval is xfs_dir2_namehash() ^ p_ino +*/ +struct xfs_parent_rec { +__be64 p_ino; +__be32 p_gen; +} __packed; +#endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_ondisk.h +1 -0
@@ -119,6 +119,7 @@ xfs_check_ondisk_structs(void)XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, offset,1 );XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, name,3 );XFS_CHECK_STRUCT_SIZE (xfs_dir2_sf_hdr_t,10 ); + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_rec, 12);/* log structures */ XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format, 88);
Re: [PATCH 06/32] xfs: define parent pointer ondisk extended attribute format
Looks good:
PATCH 7/32 xfs: allow xattr matching on name and value for local/sf attrs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a new XFS_DA_OP_PARENT flag to signal that the caller wants to look up a parent pointer extended attribute by name and value. This only works with shortform and local attributes. Only parent pointers need this functionality and parent pointers cannot be remote xattrs, so this limitation is ok for now.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 44 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +40 -4
@@ -513,12 +513,33 @@ static inline unsigned int xfs_attr_match_mask(const struct xfs_da_args *args)return XFS_ATTR_NSP_ONDISK_MASK |XFS_ATTR_INCOMPLETE ; } +static inline bool +xfs_attr_parent_match ( +const struct xfs_da_args *args, +const void *value, +unsigned int valuelen) +{ +ASSERT (args->value !=NULL ); + +/* Parent pointers do not use remote values */ +if (!value) +return false; + +/* The only value we support is a parent rec. */ +if (valuelen !=sizeof (struct xfs_parent_rec )) +return false; + +return memcmp (args->value, value, valuelen) ==0 ; +} +static bool xfs_attr_match (struct xfs_da_args *args,unsigned int attr_flags,const unsigned char *name, -unsigned int namelen) +unsigned int namelen, +const void *value, +unsigned int valuelen) {unsigned int mask =xfs_attr_match_mask (args);
@@ -529,6 +550,9 @@ xfs_attr_match(if (memcmp (args->name, name, namelen) !=0 )return false; +if (attr_flags &XFS_ATTR_PARENT ) +return xfs_attr_parent_match (args, value, valuelen); +return true; }
@@ -538,6 +562,13 @@ xfs_attr_copy_value( unsigned char *value, int valuelen) { +/* +* Parent pointer lookups require the caller to specify the name and +* value, so don't copy anything. +*/ +if (args->attr_filter &XFS_ATTR_PARENT ) +return 0 ; +/* * No copy if all we have to do is get the length */
@@ -747,7 +778,9 @@ xfs_attr_sf_findname( sfe <xfs_attr_sf_endptr (sf); sfe = xfs_attr_sf_nextentry(sfe)) {if (xfs_attr_match (args, sfe->flags, sfe->nameval, - sfe->namelen)) + sfe->namelen, + &sfe->nameval[sfe->namelen], + sfe->valuelen))return sfe; }
@@ -2444,14 +2477,17 @@ xfs_attr3_leaf_lookup_int( name_loc =xfs_attr3_leaf_name_local (leaf, probe);if (!xfs_attr_match (args, entry->flags, name_loc->nameval, - name_loc->namelen)) + name_loc->namelen, + &name_loc->nameval[name_loc->namelen], +be16_to_cpu (name_loc->valuelen)))continue ; args->index = probe;return -EEXIST ; }else { name_rmt =xfs_attr3_leaf_name_remote (leaf, probe);if (!xfs_attr_match (args, entry->flags, name_rmt->name, - name_rmt->namelen)) + name_rmt->namelen,NULL , +be32_to_cpu (name_rmt->valuelen)))continue ; args->index = probe; args->rmtvaluelen =be32_to_cpu (name_rmt->valuelen);
Re: [PATCH 07/32] xfs: allow xattr matching on name and value for local/sf attrs
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add a new
XFS_DA_OP_PARENTflag to signal that the caller wants to look
The flag doesn't actually exist, the match is done on the XFS_ATTR_PARENT namespaces.
fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr3_leaf_lookup_int( · line 2482
Quoted 9 lines
fs/xfs/libxfs/xfs_attr_leaf.c +3
-1@@ -2444,14 +2477,17 @@ xfs_attr3_leaf_lookup_int( name_loc =xfs_attr3_leaf_name_local (leaf, probe);if (!xfs_attr_match (args, entry->flags, name_loc->nameval, - name_loc->namelen)) + name_loc->namelen, + &name_loc->nameval[name_loc->namelen], +be16_to_cpu (name_loc->valuelen)))
If we'd switch from the odd pre-existing three-tab indent to the normal two-tab indent we'd avoid the overly long line here.
fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr3_leaf_lookup_int( · line 2490
Quoted 9 lines
fs/xfs/libxfs/xfs_attr_leaf.c +2
-1continue ; args->index = probe;return -EEXIST ; }else { name_rmt =xfs_attr3_leaf_name_remote (leaf, probe);if (!xfs_attr_match (args, entry->flags, name_rmt->name, - name_rmt->namelen)) + name_rmt->namelen,NULL , +be32_to_cpu (name_rmt->valuelen)))
... and here.
The remote side might also benefit from a local variable to store the endian swapped version of the valuelen instead of calculating it twice.
Otherwise looks good:
Re: [PATCH 07/32] xfs: allow xattr matching on name and value for local/sf attrs
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add a new
XFS_DA_OP_PARENTflag to signal that the caller wants to lookThe flag doesn't actually exist, the match is done on the
XFS_ATTR_PARENTnamespaces.
How about:
"xfs: allow xattr matching on name and value for local/sf pptr attrs
"If a file is hardlinked with the same name but from multiple parents, the parent pointers will all have the same dirent name (== attr name) but with different parent_ino/parent_gen values. To disambiguate, we need to be able to match on both the attr name and the attr value. This is in contrast to regular xattrs, which are matched only on name.
"Therefore, plumb in the ability to match shortform and local attrs on name and value in the XFS_ATTR_PARENT namespace. Parent pointer attr values are never large enough to be stored in a remote attr, so we need can reject these cases as corruption."
Quoted 13 lines
fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr3_leaf_lookup_int( · line 2482
Quoted 9 lines
fs/xfs/libxfs/xfs_attr_leaf.c +3
-1@@ -2444,14 +2477,17 @@ xfs_attr3_leaf_lookup_int( name_loc =xfs_attr3_leaf_name_local (leaf, probe);if (!xfs_attr_match (args, entry->flags, name_loc->nameval, - name_loc->namelen)) + name_loc->namelen, + &name_loc->nameval[name_loc->namelen], +be16_to_cpu (name_loc->valuelen)))If we'd switch from the odd pre-existing three-tab indent to the normal two-tab indent we'd avoid the overly long line here.
Quoted 11 lines
fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr3_leaf_lookup_int( · line 2490
Quoted 9 lines
fs/xfs/libxfs/xfs_attr_leaf.c +2
-1continue ; args->index = probe;return -EEXIST ; }else { name_rmt =xfs_attr3_leaf_name_remote (leaf, probe);if (!xfs_attr_match (args, entry->flags, name_rmt->name, - name_rmt->namelen)) + name_rmt->namelen,NULL , +be32_to_cpu (name_rmt->valuelen)))... and here.
Believe it or not that's what vim autoformat does by default. But yes, I'll reduce it to two indents to reduce the indentation and overflow.
Quoted 2 lines
The remote side might also benefit from a local variable to store the endian swapped version of the valuelen instead of calculating it twice.
Ok.
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
Re: [PATCH 07/32] xfs: allow xattr matching on name and value for local/sf attrs
Quoted 3 lines · Darrick J. Wong
How about:
"xfs: allow xattr matching on name and value for local/sf pptr attrs
How about: "match on the attr value as well for parent pointers"
for the subject?
The commit message body looks good to me.
PATCH 8/32 xfs: allow logged xattr operations if parent pointers are enabled
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Don't trip this assertion about attr log items if we have parent pointers enabled. Parent pointers are an incompat feature that doesn't use any of the functionality protected by XFS_SB_FEAT_INCOMPAT_LOG_XATTRS, which is why this is ok.
--- fs/xfs/xfs_attr_item.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/xfs/xfs_attr_item.c +1 -1
@@ -474,7 +474,7 @@ xfs_attri_validate( {unsigned int op =xfs_attr_log_item_op (attrp); -if (!xfs_is_using_logged_xattrs (mp)) +if (!xfs_is_using_logged_xattrs (mp) && !xfs_has_parent (mp))return false;if (attrp->__pad !=0 )
Re: [PATCH 08/32] xfs: allow logged xattr operations if parent pointers are enabled
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Don't trip this assertion about attr log items if we have parent pointers enabled. Parent pointers are an incompat feature that doesn't use any of the functionality protected by
XFS_SB_FEAT_INCOMPAT_LOG_XATTRS, which is why this is ok.
I'd move the checks into the switch on op below, so that we check the log attrs feature for the "normal" logged attrs and the parent pointers flag for the parent pointer ops.
Re: [PATCH 08/32] xfs: allow logged xattr operations if parent pointers are enabled
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Don't trip this assertion about attr log items if we have parent pointers enabled. Parent pointers are an incompat feature that doesn't use any of the functionality protected by
XFS_SB_FEAT_INCOMPAT_LOG_XATTRS, which is why this is ok.I'd move the checks into the switch on op below, so that we check the log attrs feature for the "normal" logged attrs and the parent pointers flag for the parent pointer ops.
Good idea.
--D
PATCH 9/32 xfs: log parent pointer xattr removal operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The parent pointer code needs to do a deferred parent pointer remove operation with the xattr log intent code. Declare a new logged xattr opcode and push it through the log.
--- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_attr_item.c | 53 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_attr_item.h | 2 ++ 3 files changed, 56 insertions(+)
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -1026,6 +1026,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_SET 1/* Set the attribute */ #define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ +#define XFS_ATTRI_OP_FLAGS_PPTR_REMOVE 5/* Remove parent pointer */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
fs/xfs/xfs_attr_item.c +53 -0
89 lines not on this page · raw message
fs/xfs/xfs_attr_item.h +2 -0
@@ -58,5 +58,7 @@ enum xfs_attr_defer_op { };void xfs_attr_defer_add (struct xfs_da_args *args,enum xfs_attr_defer_op op); +void xfs_attr_defer_parent (struct xfs_da_args *args, +enum xfs_attr_defer_op op);#endif /* __XFS_ATTR_ITEM_H__ */
Re: [PATCH 09/32] xfs: log parent pointer xattr removal operations
Looks good:
PATCH 10/32 xfs: log parent pointer xattr setting operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The parent pointer code needs to do a deferred parent pointer set operation with the xattr log intent code. Declare a new logged xattr opcode and push it through the log.
--- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_attr_item.c | 9 ++++++++- 2 files changed, 9 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -1026,6 +1026,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_SET 1/* Set the attribute */ #define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ +#define XFS_ATTRI_OP_FLAGS_PPTR_SET 4/* Set parent pointer */ #define XFS_ATTRI_OP_FLAGS_PPTR_REMOVE 5/* Remove parent pointer */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */
fs/xfs/xfs_attr_item.c +8 -1
@@ -491,6 +491,7 @@ xfs_attri_validate(return false;switch (op) { +case XFS_ATTRI_OP_FLAGS_PPTR_SET :case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE :if (!xfs_has_parent (mp))return false;
@@ -595,6 +596,7 @@ xfs_attri_recover_work(xfs_attr_sethash (args);switch (xfs_attr_intent_op (attr)) { +case XFS_ATTRI_OP_FLAGS_PPTR_SET :case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : args->total =xfs_attr_calc_size (args, &local);
@@ -778,6 +780,9 @@ xfs_attr_defer_parent(switch (op) {case XFS_ATTR_DEFER_SET : + new->xattri_op_flags =XFS_ATTRI_OP_FLAGS_PPTR_SET ; + new->xattri_dela_state =xfs_attr_init_add_state (args); +break ;case XFS_ATTR_DEFER_REPLACE :/* will be added in subsequent patches */ ASSERT (0 );
@@ -841,6 +846,7 @@ xlog_recover_attri_commit_pass2( op =xfs_attr_log_item_op (attri_formatp);switch (op) {case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE : +case XFS_ATTRI_OP_FLAGS_PPTR_SET :/* Log item, attr name, attr value */ if (item->ri_total !=3 ) { XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
@@ -926,6 +932,7 @@ xlog_recover_attri_commit_pass2( } fallthrough;case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE : +case XFS_ATTRI_OP_FLAGS_PPTR_SET :case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :/*
@@ -933,7 +940,7 @@ xlog_recover_attri_commit_pass2(* and do not take a newname. Values are optional for set and * replace. * -* Name-value remove operations must have a name, do not +* Name-value set/remove operations must have a name, do not * take a newname, and can take a value. */ if (attr_name ==NULL || name_len ==0 ) {
PATCH 11/32 xfs: log parent pointer xattr replace operations
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
The parent pointer code needs to do a deferred parent pointer replace operation with the xattr log intent code. Declare a new logged xattr opcode and push it through the log.
(Formerly titled "xfs: Add new name to attri/d" and described as follows:
This patch adds two new fields to the atti/d. They are nname and nnamelen. This will be used for parent pointer updates since a rename operation may cause the parent pointer to update both the name and value. So we need to carry both the new name as well as the target name in the attri/d.)
--- fs/xfs/libxfs/xfs_attr.c | 19 ++++ fs/xfs/libxfs/xfs_attr.h | 4 - fs/xfs/libxfs/xfs_da_btree.h | 4 + fs/xfs/libxfs/xfs_log_format.h | 20 ++++ fs/xfs/xfs_attr_item.c | 193 ++++++++++++++++++++++++++++++++++++---- fs/xfs/xfs_attr_item.h | 2 6 files changed, 218 insertions(+), 24 deletions(-)
fs/xfs/libxfs/xfs_attr.c +19 -0
@@ -441,6 +441,23 @@ xfs_attr_hashval(return xfs_attr_hashname (name, namelen); } +/* +* PPTR_REPLACE operations require the caller to set the old and new names and +* values explicitly. Update the canonical fields to the new name and value +* here now that the removal phase has finished. +*/ +static void +xfs_attr_update_pptr_replace_args ( +struct xfs_da_args *args) +{ +ASSERT (args->new_namelen >0 ); + args->name = args->new_name; + args->namelen = args->new_namelen; + args->value = args->new_value; + args->valuelen = args->new_valuelen; +xfs_attr_sethash (args); +} +/* * Handle the state change on completion of a multi-state attr operation. *
@@ -461,6 +478,8 @@ xfs_attr_complete_op(if (!(args->op_flags &XFS_DA_OP_REPLACE )) replace_state =XFS_DAS_DONE ; +else if (xfs_attr_intent_op (attr) ==XFS_ATTRI_OP_FLAGS_PPTR_REPLACE ) +xfs_attr_update_pptr_replace_args (args); args->op_flags &= ~XFS_DA_OP_REPLACE ; args->attr_filter &= ~XFS_ATTR_INCOMPLETE ;
fs/xfs/libxfs/xfs_attr.h +2 -2
@@ -510,8 +510,8 @@ struct xfs_attr_intent {struct xfs_da_args *xattri_da_args;/* -* Shared buffer containing the attr name and value so that the logging -* code can share large memory buffers between log items. +* Shared buffer containing the attr name, new name, and value so that +* the logging code can share large memory buffers between log items. */ struct xfs_attri_log_nameval *xattri_nameval;
fs/xfs/libxfs/xfs_da_btree.h +4 -0
@@ -55,7 +55,9 @@ enum xfs_dacmp { typedef struct xfs_da_args { struct xfs_da_geometry *geo; /* da block geometry */ const uint8_t *name; /* string (maybe not NULL terminated) */ + const uint8_t *new_name; /* new attr name */ void *value; /* set of bytes (maybe contain NULLs) */ + void *new_value; /* new xattr value (may contain NULLs) */ struct xfs_inode *dp; /* directory inode to manipulate */ struct xfs_trans *trans; /* current trans (changes over time) */
@@ -63,11 +65,13 @@ typedef struct xfs_da_args {xfs_ino_t owner;/* inode that owns the dir/attr data */ int valuelen;/* length of value */ +int new_valuelen;/* length of new_value */ uint8_t filetype;/* filetype of inode for directories */ uint8_t op_flags;/* operation flags */ uint8_t attr_filter;/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ uint8_t xattr_flags;/* XATTR_{CREATE,REPLACE} */ short namelen;/* length of string (maybe no NULL) */ +short new_namelen;/* length of new attr name */ xfs_dahash_t hashval;/* hash value of name */ xfs_extlen_t total;/* total blocks needed, for 1st bmap */ int whichfork;/* data or attribute fork */
fs/xfs/libxfs/xfs_log_format.h +17 -3
@@ -115,11 +115,13 @@ struct xfs_unmount_log_format {#define XLOG_REG_TYPE_BUD_FORMAT 26#define XLOG_REG_TYPE_ATTRI_FORMAT 27#define XLOG_REG_TYPE_ATTRD_FORMAT 28 -#define XLOG_REG_TYPE_ATTR_NAME 29 +#define XLOG_REG_TYPE_ATTR_NAME 29#define XLOG_REG_TYPE_ATTR_VALUE 30#define XLOG_REG_TYPE_XMI_FORMAT 31#define XLOG_REG_TYPE_XMD_FORMAT 32 -#define XLOG_REG_TYPE_MAX 32 +#define XLOG_REG_TYPE_ATTR_NEWNAME 33 +#define XLOG_REG_TYPE_ATTR_NEWVALUE 34 +#define XLOG_REG_TYPE_MAX 34/* * Flags to log operation header
@@ -1028,6 +1030,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ #define XFS_ATTRI_OP_FLAGS_PPTR_SET 4/* Set parent pointer */ #define XFS_ATTRI_OP_FLAGS_PPTR_REMOVE 5/* Remove parent pointer */ +#define XFS_ATTRI_OP_FLAGS_PPTR_REPLACE 6/* Replace parent pointer */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
@@ -1050,7 +1053,18 @@ struct xfs_attri_log_format {uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */ uint32_t alfi_op_flags;/* marks the op as a set or remove */ -uint32_t alfi_name_len;/* attr name length */ +union { +uint32_t alfi_name_len;/* attr name length */ +struct { +/* +* For PPTR_REPLACE, these are the lengths of the old +* and new attr names. The new and old values must +* have the same length. +*/ +uint16_t alfi_old_name_len; +uint16_t alfi_new_name_len; + }; + };uint32_t alfi_value_len;/* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */ };
fs/xfs/xfs_attr_item.c +174 -19
334 lines not on this page · raw message
fs/xfs/xfs_attr_item.h +2 -0
@@ -13,7 +13,9 @@ struct kmem_zone;struct xfs_attri_log_nameval {struct xfs_log_iovec name; +struct xfs_log_iovec new_name;/* PPTR_REPLACE only */ struct xfs_log_iovec value; +struct xfs_log_iovec new_value;/* PPTR_REPLACE only */ refcount_t refcount;/* name and value follow the end of this struct */
Re: [PATCH 11/32] xfs: log parent pointer xattr replace operations
Quoted 8 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
The parent pointer code needs to do a deferred parent pointer replace operation with the xattr log intent code. Declare a new logged xattr opcode and push it through the log.
(Formerly titled "xfs: Add new name to attri/d" and described as follows:
I don't think this history is very important. The being said, I suspect this and the previous two patches should be combined into a single one adding the on-disk formats for parent pointers, and the commit log could use a complete rewrite saying that it a
fs/xfs/xfs_attr_item.c xfs_attri_validate( · line 584
Quoted 7 lines
fs/xfs/xfs_attr_item.c +7
-0+return false; +if (attrp->alfi_old_name_len ==0 || + attrp->alfi_old_name_len >XATTR_NAME_MAX ) +return false; +if (attrp->alfi_new_name_len ==0 || + attrp->alfi_new_name_len >XATTR_NAME_MAX ) +return false;
Given that we have four copies of this (arguably simple) check, should we grow a helper for it?
fs/xfs/xfs_attr_item.c xfs_attri_validate( · line 587
Quoted 3 lines
fs/xfs/xfs_attr_item.c +3
-0+if (attrp->alfi_value_len ==0 || + attrp->alfi_value_len >XATTR_SIZE_MAX ) +return false;
All parent pointer attrs must be sized for exactly the parent_rec, so we should probably check for that explicitly?
fs/xfs/xfs_attr_item.c xfs_attr_relog_intent( · line 788
Quoted 1 line
fs/xfs/xfs_attr_item.c +1
-0+if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_PPTR_REPLACE ) {
Please avoid the overly long line here.
fs/xfs/xfs_attr_item.c xlog_recover_attri_commit_pass2( · line 1009
Quoted 4 lines
fs/xfs/xfs_attr_item.c +3
-0+/* Validate the new attr name */ +if (new_name_len >0 ) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (new_name_len)) {
.. and here.
And while we're at it, maybe factor the checking for valid xattr name and value log iovecs into little helper instead of duplicating them a few times?
Re: [PATCH 11/32] xfs: log parent pointer xattr replace operations
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
The parent pointer code needs to do a deferred parent pointer replace operation with the xattr log intent code. Declare a new logged xattr opcode and push it through the log.
Quoted 7 lines
Quoted 2 lines
(Formerly titled "xfs: Add new name to attri/d" and described as follows:
I don't think this history is very important. The being said, I suspect this and the previous two patches should be combined into a single one adding the on-disk formats for parent pointers, and the commit log could use a complete rewrite saying that it a
I combined the three patches into this:
xfs: create attr log item opcodes and formats for parent pointers
Make the necessary alterations to the extended attribute log intent item ondisk format so that we can log parent pointer operations. This requires the creation of new opcodes specific to parent pointers, and a new four-argument replace operation to handle renames. At this point this part of the patchset has changed so much from what Allison original wrote that I no longer think her SoB applies.
and about time, I was getting real irritated at having to iterate back and forth across those. ;)
Quoted 10 lines
fs/xfs/xfs_attr_item.c xfs_attri_validate( · line 584
Quoted 7 lines
fs/xfs/xfs_attr_item.c +7
-0+return false; +if (attrp->alfi_old_name_len ==0 || + attrp->alfi_old_name_len >XATTR_NAME_MAX ) +return false; +if (attrp->alfi_new_name_len ==0 || + attrp->alfi_new_name_len >XATTR_NAME_MAX ) +return false;Given that we have four copies of this (arguably simple) check, should we grow a helper for it?
static inline boolxfs_attri_validate_namelen(unsigned int namelen)
{
return namelen > 0 && namelen <= XATTR_NAME_MAX;
}Done.
Quoted 6 lines
fs/xfs/xfs_attr_item.c xfs_attri_validate( · line 587
Quoted 3 lines
fs/xfs/xfs_attr_item.c +3
-0+if (attrp->alfi_value_len ==0 || + attrp->alfi_value_len >XATTR_SIZE_MAX ) +return false;All parent pointer attrs must be sized for exactly the
parent_rec, so we should probably check for that explicitly?
Done.
Quoted 3 lines
fs/xfs/xfs_attr_item.c xfs_attr_relog_intent( · line 788
Quoted 1 line
fs/xfs/xfs_attr_item.c +1
-0+if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_PPTR_REPLACE ) {Please avoid the overly long line here.
I've turned that into a switch()
Quoted 10 lines
fs/xfs/xfs_attr_item.c xlog_recover_attri_commit_pass2( · line 1009
Quoted 4 lines
fs/xfs/xfs_attr_item.c +3
-0+/* Validate the new attr name */ +if (new_name_len >0 ) { +if (item->ri_buf[i].i_len !=xlog_calc_iovec_len (new_name_len)) {.. and here.
And while we're at it, maybe factor the checking for valid xattr name and value log iovecs into little helper instead of duplicating them a few times?
Ok, I'll add that as a prep patch.
--D
Re: [PATCH 11/32] xfs: log parent pointer xattr replace operations
Quoted 8 lines · Darrick J. Wong
xfs: create attr log item opcodes and formats for parent pointers
Make the necessary alterations to the extended attribute log intent item ondisk format so that we can log parent pointer operations. This requires the creation of new opcodes specific to parent pointers, and a new four-argument replace operation to handle renames. At this point this part of the patchset has changed so much from what Allison original wrote that I no longer think her SoB applies.
Sounds good.
PATCH 12/32 xfs: record inode generation in xattr update log intent items
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
For parent pointer updates, record the i_generation of the file that is being updated so that we don't accidentally jump generations.
--- fs/xfs/libxfs/xfs_log_format.h | 2 +- fs/xfs/xfs_attr_item.c | 26 ++++++++++++++++++++++++-- 2 files changed, 25 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +1 -1
@@ -1049,7 +1049,7 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t __pad;/* pad to 64 bit aligned */ +uint32_t alfi_igen;/* generation of alfi_ino for pptr ops */ uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */ uint32_t alfi_op_flags;/* marks the op as a set or remove */
fs/xfs/xfs_attr_item.c +24 -2
@@ -382,14 +382,22 @@ xfs_attr_log_item( attrp->alfi_op_flags = attr->xattri_op_flags; attrp->alfi_value_len = attr->xattri_nameval->value.i_len; -if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_PPTR_REPLACE ) { +switch (xfs_attr_log_item_op (attrp)) { +case XFS_ATTRI_OP_FLAGS_PPTR_REPLACE :ASSERT (attr->xattri_nameval->value.i_len == attr->xattri_nameval->new_value.i_len); + attrp->alfi_igen =VFS_I (attr->xattri_da_args->dp)->i_generation; attrp->alfi_old_name_len = attr->xattri_nameval->name.i_len; attrp->alfi_new_name_len = attr->xattri_nameval->new_name.i_len; - }else { +break ; +case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE : +case XFS_ATTRI_OP_FLAGS_PPTR_SET : + attrp->alfi_igen =VFS_I (attr->xattri_da_args->dp)->i_generation; + fallthrough; +default : attrp->alfi_name_len = attr->xattri_nameval->name.i_len; +break ; }ASSERT (!(attr->xattri_da_args->attr_filter & ~XFS_ATTRI_FILTER_MASK ));
@@ -632,6 +640,19 @@ xfs_attri_recover_work(if (error)return ERR_PTR (error); +switch (xfs_attr_log_item_op (attrp)) { +case XFS_ATTRI_OP_FLAGS_PPTR_SET : +case XFS_ATTRI_OP_FLAGS_PPTR_REPLACE : +case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE : +if (VFS_I (ip)->i_generation != attrp->alfi_igen) { +xfs_irele (ip); +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + attrp,sizeof (*attrp)); +return ERR_PTR (-EFSCORRUPTED ); + } +break ; + } +if (xfs_inode_has_attr_fork (ip)) { error =xfs_attri_iread_extents (ip);if (error) {
@@ -782,6 +803,7 @@ xfs_attr_relog_intent( new_attrp = &new_attrip->attri_format; new_attrp->alfi_ino = old_attrp->alfi_ino; + new_attrp->alfi_igen = old_attrp->alfi_igen; new_attrp->alfi_op_flags = old_attrp->alfi_op_flags; new_attrp->alfi_value_len = old_attrp->alfi_value_len;
Re: [PATCH 12/32] xfs: record inode generation in xattr update log intent items
fs/xfs/xfs_attr_item.c xfs_attr_log_item( · line 390
Quoted 7 lines
fs/xfs/xfs_attr_item.c +3
-1-if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_PPTR_REPLACE ) { +switch (xfs_attr_log_item_op (attrp)) { +case XFS_ATTRI_OP_FLAGS_PPTR_REPLACE :ASSERT (attr->xattri_nameval->value.i_len == attr->xattri_nameval->new_value.i_len); + attrp->alfi_igen =VFS_I (attr->xattri_da_args->dp)->i_generation;
Please avoid the overly long lines (maybe xattri_da_args needs a shortet name or we want a local variable for it?)
PATCH 13/32 xfs: Expose init_xattrs in xfs_create_tmpfile
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Tmp files are used as part of rename operations and will need attr forks initialized for parent pointers. Expose the init_xattrs parameter to the calling function to initialize the fork.
--- fs/xfs/xfs_inode.c | 5 +++-- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_iops.c | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +3 -2
@@ -1184,6 +1184,7 @@ xfs_create_tmpfile(struct mnt_idmap *idmap, struct xfs_inode *dp, umode_t mode, + bool init_xattrs, struct xfs_inode **ipp) { struct xfs_mount *mp = dp->i_mount;
@@ -1224,7 +1225,7 @@ xfs_create_tmpfile( error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error =xfs_init_new_inode (idmap, tp, dp, ino, mode, -0 ,0 , prid, false, &ip); +0 ,0 , prid, init_xattrs, &ip);if (error) goto out_trans_cancel;
@@ -3036,7 +3037,7 @@ xfs_rename_alloc_whiteout(int error; error =xfs_create_tmpfile (idmap, dp,S_IFCHR |WHITEOUT_MODE , - &tmpfile); + false, &tmpfile);if (error)return error;
fs/xfs/xfs_inode.h +1 -1
@@ -522,7 +522,7 @@ int xfs_create(struct mnt_idmap *idmap, umode_t mode, dev_t rdev, bool need_xattr, struct xfs_inode **ipp);int xfs_create_tmpfile (struct mnt_idmap *idmap, -struct xfs_inode *dp,umode_t mode, +struct xfs_inode *dp,umode_t mode,bool init_xattrs,struct xfs_inode **ipp);int xfs_remove (struct xfs_inode *dp,struct xfs_name *name,struct xfs_inode *ip);
fs/xfs/xfs_iops.c +1 -1
@@ -201,7 +201,7 @@ xfs_generic_create(xfs_create_need_xattr (dir, default_acl, acl), &ip); }else { - error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, &ip); + error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, false, &ip); }if (unlikely (error)) goto out_free_acl;
Re: [PATCH 13/32] xfs: Expose init_xattrs in xfs_create_tmpfile
Looks good:
PATCH 14/32 xfs: add parent pointer validator functions
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings. So we need to modify attr_namecheck to verify parent pointer records when the XFS_ATTR_PARENT flag is set. At the same time, we need to validate attr values during log recovery if the xattr is really a parent pointer.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_attr.c | 5 ++ fs/xfs/libxfs/xfs_parent.c | 92 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 15 +++++++ fs/xfs/xfs_attr_item.c | 15 +++++++ 5 files changed, 128 insertions(+) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h
fs/xfs/Makefile +1 -0
@@ -42,6 +42,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_buf.o \ xfs_log_rlimit.o \ xfs_ag_resv.o \ + xfs_parent.o \ xfs_rmap.o \ xfs_rmap_btree.o \ xfs_refcount.o \
fs/xfs/libxfs/xfs_attr.c +5 -0
@@ -26,6 +26,7 @@#include "xfs_trace.h" #include "xfs_attr_item.h" #include "xfs_xattr.h" +#include "xfs_parent.h" struct kmem_cache *xfs_attr_intent_cache;
@@ -1570,6 +1571,10 @@ xfs_attr_namecheck(if (length >=MAXNAMELEN )return false; +/* Parent pointers have their own validation. */ +if (attr_flags &XFS_ATTR_PARENT ) +return xfs_parent_namecheck (attr_flags, name, length); +/* There shouldn't be any nulls here */ return !memchr (name,0 , length); }
fs/xfs/libxfs/xfs_parent.c new file +92 -0
92 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022-2024 Oracle. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* Metadata validators */ +bool xfs_parent_namecheck (unsigned int attr_flags,const void *name, +size_t length); +bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value, +size_t valuelen); + +#endif /* __XFS_PARENT_H__ */
fs/xfs/xfs_attr_item.c +15 -0
@@ -27,6 +27,7 @@#include "xfs_error.h" #include "xfs_log_priv.h" #include "xfs_log_recover.h" +#include "xfs_parent.h" struct kmem_cache *xfs_attri_cache;struct kmem_cache *xfs_attrd_cache;
@@ -1055,6 +1056,13 @@ xlog_recover_attri_commit_pass2( } attr_value = item->ri_buf[i].i_addr; +if ((attri_formatp->alfi_attr_filter &XFS_ATTR_PARENT ) && + !xfs_parent_valuecheck (mp, attr_value, value_len)) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + item->ri_buf[i].i_addr, + item->ri_buf[i].i_len); +return -EFSCORRUPTED ; + } i++; }
@@ -1067,6 +1075,13 @@ xlog_recover_attri_commit_pass2( } attr_new_value = item->ri_buf[i].i_addr; +if ((attri_formatp->alfi_attr_filter &XFS_ATTR_PARENT ) && + !xfs_parent_valuecheck (mp, attr_new_value, new_value_len)) { +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, + item->ri_buf[i].i_addr, + item->ri_buf[i].i_len); +return -EFSCORRUPTED ; + } i++; }
Re: [PATCH 14/32] xfs: add parent pointer validator functions
Quoted 3 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings.
They are now. The rest of the commit log also doesn't match the code anymore. The code itself looks good, though.
Darrick J. Wong
Reviewed-byRe: [PATCH 14/32] xfs: add parent pointer validator functions
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings.
They are now. The rest of the commit log also doesn't match the code anymore. The code itself looks good, though.
How about this, then:
xfs: add parent pointer validator functions
The attr name of a parent pointer is a string, and the attr value of a parent pointer is (more or less) a file handle. So we need to modify attr_namecheck to verify the parent pointer name, and add a xfs_parent_valuecheck function to sanitize the handle. At the same time, we need to validate attr values during log recovery if the xattr is really a parent pointer.
--D
Christoph Hellwig
Reviewed-byRe: [PATCH 14/32] xfs: add parent pointer validator functions
Quoted 10 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings.
They are now. The rest of the commit log also doesn't match the code anymore. The code itself looks good, though.
How about this, then:
Looks good:
PATCH 15/32 xfs: extend transaction reservations for parent attributes
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.
--- fs/xfs/libxfs/xfs_trans_resv.c | 326 ++++++++++++++++++++++++++++++++++------ 1 file changed, 274 insertions(+), 52 deletions(-)
fs/xfs/libxfs/xfs_trans_resv.c +274 -52
@@ -20,6 +20,9 @@#include "xfs_qm.h" #include "xfs_trans_space.h" #include "xfs_rtbitmap.h" +#include "xfs_attr_item.h" +#include "xfs_log.h" +#include "xfs_da_format.h" #define _ALLOC true#define _FREE false
@@ -422,29 +425,110 @@ xfs_calc_itruncate_reservation_minlogsize(return xfs_calc_itruncate_reservation (mp, true); } +static inline unsigned int xfs_calc_pptr_link_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_rec )) + +xlog_calc_iovec_len (MAXNAMELEN -1 ); +} +static inline unsigned int xfs_calc_pptr_unlink_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_rec )) + +xlog_calc_iovec_len (MAXNAMELEN -1 ); +} +static inline unsigned int xfs_calc_pptr_replace_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_rec )) + +xlog_calc_iovec_len (MAXNAMELEN -1 ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_rec )) + +xlog_calc_iovec_len (MAXNAMELEN -1 ); +} +/* * In renaming a files we can modify: * the five inodes involved: 5 * inode size * the two directory btrees: 2 * (max depth + v2) * dir block size * the two directory bmap btrees: 2 * max depth * block size * And the bmap_finish transaction can free dir and bmap blocks (two sets -* of bmap blocks) giving: +* of bmap blocks) giving (t2): * the agf for the ags in which the blocks live: 3 * sector size * the agfl for the ags in which the blocks live: 3 * sector size * the superblock for the free block count: sector size * the allocation btrees: 3 exts * 2 trees * (2 * max depth - 1) * block size +* If parent pointers are enabled (t3), then each transaction in the chain +* must be capable of setting or removing the extended attribute +* containing the parent information. It must also be able to handle +* the three xattr intent items that track the progress of the parent +* pointer update. */ STATIC uintxfs_calc_rename_reservation (struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max ((xfs_calc_inode_res (mp,5 ) + -xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_inode_res (mp,5 ) + +xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), +XFS_FSB_TO_B (mp,1 )); + + t2 =xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { +unsigned int rename_overhead, exchange_overhead; + + t3 =max (resp->tr_attrsetm.tr_logres, + resp->tr_attrrm.tr_logres); + +/* +* For a standard rename, the three xattr intent log items +* are (1) replacing the pptr for the source file; (2) +* removing the pptr on the dest file; and (3) adding a +* pptr for the whiteout file in the src dir. +* +* For an RENAME_EXCHANGE, there are two xattr intent +* items to replace the pptr for both src and dest +* files. Link counts don't change and there is no +* whiteout. +* +* In the worst case we can end up relogging all log +* intent items to allow the log tail to move ahead, so +* they become overhead added to each transaction in a +* processing chain. +*/ + rename_overhead =xfs_calc_pptr_replace_overhead () + +xfs_calc_pptr_unlink_overhead () + +xfs_calc_pptr_link_overhead (); + exchange_overhead =2 *xfs_calc_pptr_replace_overhead (); + + overhead +=max (rename_overhead, exchange_overhead); + } + +return overhead +max3 (t1, t2, t3); +} + +static inline unsigned int +xfs_rename_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +/* One for the rename, one more for freeing blocks */ +unsigned int ret =XFS_RENAME_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to remove or add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret +=max (resp->tr_attrsetm.tr_logcount, + resp->tr_attrrm.tr_logcount); + +return ret; }/*
@@ -461,6 +545,23 @@ xfs_calc_iunlink_remove_reservation(2 *M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_link_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_LINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * For creating a link to an inode: * the parent directory inode: inode size
@@ -477,14 +578,23 @@ STATIC uintxfs_calc_link_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_remove_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_remove_reservation (mp); + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -499,6 +609,23 @@ xfs_calc_iunlink_add_reservation(xfs_mount_t *mp)M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_remove_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_REMOVE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrrm.tr_logcount; + +return ret; +} +/* * For removing a directory entry we can modify: * the parent directory inode: inode size
@@ -515,14 +642,24 @@ STATIC uintxfs_calc_remove_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_add_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_add_reservation (mp); + + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrrm.tr_logres; + overhead +=xfs_calc_pptr_unlink_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -571,12 +708,40 @@ xfs_calc_icreate_resv_alloc(xfs_calc_finobt_res (mp); } +static inline unsigned int +xfs_icreate_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_CREATE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} + STATIC uint -xfs_calc_icreate_reservation (xfs_mount_t *mp) +xfs_calc_icreate_reservation ( +struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max (xfs_calc_icreate_resv_alloc (mp), -xfs_calc_create_resv_modify (mp)); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_icreate_resv_alloc (mp); + t2 =xfs_calc_create_resv_modify (mp); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }STATIC uint
@@ -589,6 +754,23 @@ xfs_calc_create_tmpfile_reservation(return res +xfs_calc_iunlink_add_reservation (mp); } +static inline unsigned int +xfs_mkdir_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_MKDIR_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * Making a new directory is the same as creating a new file. */
@@ -599,6 +781,22 @@ xfs_calc_mkdir_reservation(return xfs_calc_icreate_reservation (mp); } +static inline unsigned int +xfs_symlink_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_SYMLINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +}/* * Making a new symplink is the same as creating a new file, but
@@ -911,6 +1109,52 @@ xfs_calc_sb_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize); } +/* +* Namespace reservations. +* +* These get tricky when parent pointers are enabled as we have attribute +* modifications occurring from within these transactions. Rather than confuse +* each of these reservation calculations with the conditional attribute +* reservations, add them here in a clear and concise manner. This requires that +* the attribute reservations have already been calculated. +* +* Note that we only include the static attribute reservation here; the runtime +* reservation will have to be modified by the size of the attributes being +* added/removed/modified. See the comments on the attribute reservation +* calculations for more details. +*/ +STATIC void +xfs_calc_namespace_reservations ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +ASSERT (resp->tr_attrsetm.tr_logres >0 ); + + resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); + resp->tr_rename.tr_logcount =xfs_rename_log_count (mp, resp); + resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); + resp->tr_link.tr_logcount =xfs_link_log_count (mp, resp); + resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); + resp->tr_remove.tr_logcount =xfs_remove_log_count (mp, resp); + resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); + resp->tr_symlink.tr_logcount =xfs_symlink_log_count (mp, resp); + resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); + resp->tr_create.tr_logcount =xfs_icreate_log_count (mp, resp); + resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); + resp->tr_mkdir.tr_logcount =xfs_mkdir_log_count (mp, resp); + resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +} + void xfs_trans_resv_calc( struct xfs_mount *mp,
@@ -930,35 +1174,11 @@ xfs_trans_resv_calc( resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); - resp->tr_rename.tr_logcount =XFS_RENAME_LOG_COUNT ; - resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); - resp->tr_link.tr_logcount =XFS_LINK_LOG_COUNT ; - resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); - resp->tr_remove.tr_logcount =XFS_REMOVE_LOG_COUNT ; - resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); - resp->tr_symlink.tr_logcount =XFS_SYMLINK_LOG_COUNT ; - resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); - resp->tr_create.tr_logcount =XFS_CREATE_LOG_COUNT ; - resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_create_tmpfile.tr_logres =xfs_calc_create_tmpfile_reservation (mp); resp->tr_create_tmpfile.tr_logcount =XFS_CREATE_TMPFILE_LOG_COUNT ; resp->tr_create_tmpfile.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); - resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; - resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_ifree.tr_logres =xfs_calc_ifree_reservation (mp); resp->tr_ifree.tr_logcount =XFS_INACTIVE_LOG_COUNT ; resp->tr_ifree.tr_logflags |=XFS_TRANS_PERM_LOG_RES ;
@@ -988,6 +1208,8 @@ xfs_trans_resv_calc( resp->tr_qm_dqalloc.tr_logcount =XFS_WRITE_LOG_COUNT ; resp->tr_qm_dqalloc.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +xfs_calc_namespace_reservations (mp, resp); +/* * The following transactions are logged in logical format with * a default log count.
Re: [PATCH 15/32] xfs: extend transaction reservations for parent attributes
Looks good:
PATCH 16/32 xfs: create a hashname function for parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Although directory entry and parent pointer recordsets look very similar (name -> ino), there's one major difference between them: a file can be hardlinked from multiple parent directories with the same filename. This is common in shared container environments where a base directory tree might be hardlink-copied multiple times. IOWs the same 'ls' program might be hardlinked to multiple /srv/*/bin/ls paths.
We don't want parent pointer operations to bog down on hash collisions between the same dirent name, so create a special hash function that mixes in the parent directory inode number.
--- fs/xfs/libxfs/xfs_attr.c | 3 +++ fs/xfs/libxfs/xfs_parent.c | 49 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 5 ++++ fs/xfs/scrub/attr.c | 4 ++++ 4 files changed, 61 insertions(+)
fs/xfs/libxfs/xfs_attr.c +3 -0
@@ -439,6 +439,9 @@ xfs_attr_hashval( {ASSERT (xfs_attr_check_namespace (attr_flags)); +if (attr_flags &XFS_ATTR_PARENT ) +return xfs_parent_hashattr (mp, name, namelen, value, valuelen); +return xfs_attr_hashname (name, namelen); }
fs/xfs/libxfs/xfs_parent.c +49 -0
52 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +5 -0
@@ -12,4 +12,9 @@ bool xfs_parent_namecheck(unsigned int attr_flags, const void *name,bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value,size_t valuelen); +xfs_dahash_t xfs_parent_hashval (struct xfs_mount *mp,const uint8_t *name, +int namelen,xfs_ino_t parent_ino); +xfs_dahash_t xfs_parent_hashattr (struct xfs_mount *mp,const uint8_t *name, +int namelen,const void *value,int valuelen); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/attr.c +4 -0
@@ -536,6 +536,10 @@ xchk_xattr_rec(xchk_da_set_corrupt (ds, level); goto out; } +if (ent->flags &XFS_ATTR_PARENT ) { +xchk_da_set_corrupt (ds, level); + goto out; + } calc_hash =xfs_attr_hashval (mp, ent->flags, rentry->name, rentry->namelen,NULL ,be32_to_cpu (rentry->valuelen));
Re: [PATCH 16/32] xfs: create a hashname function for parent pointers
fs/xfs/libxfs/xfs_parent.c xfs_parent_valuecheck( · line 115
Quoted 8 lines
fs/xfs/libxfs/xfs_parent.c +8
-0+/* +* Use the same dirent name hash as would be used on the directory, but +* mix in the parent inode number. +*/ + ret =xfs_dir2_hashname (mp, &xname); + ret ^=upper_32_bits (parent_ino); + ret ^=lower_32_bits (parent_ino); +return ret;
Totally superficial nit, but wouldn't this read a little nicer as:
return xfs_dir2_hashname(mp, &xname) ^ lower_32_bits(parent_ino) ^ upper_32_bits(parent_ino);
?
Otherwise looks good:
Re: [PATCH 16/32] xfs: create a hashname function for parent pointers
Quoted 11 lines · Christoph Hellwig
fs/xfs/libxfs/xfs_parent.c xfs_parent_valuecheck( · line 115
Quoted 8 lines
fs/xfs/libxfs/xfs_parent.c +8
-0+/* +* Use the same dirent name hash as would be used on the directory, but +* mix in the parent inode number. +*/ + ret =xfs_dir2_hashname (mp, &xname); + ret ^=upper_32_bits (parent_ino); + ret ^=lower_32_bits (parent_ino); +return ret;Totally superficial nit, but wouldn't this read a little nicer as:
Quoted 5 lines
return xfs_dir2_hashname(mp, &xname) ^ lower_32_bits(parent_ino) ^ upper_32_bits(parent_ino);?
Yeah, will change.
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
PATCH 17/32 xfs: parent pointer attribute creation
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add parent pointer attribute during xfs_create, and subroutines to initialize attributes
[djwong: shorten names, adjust to new format, set init_xattrs for parent pointers]
--- fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_parent.c | 68 +++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 65 +++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_trans_space.c | 52 ++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 9 +++-- fs/xfs/scrub/tempfile.c | 2 + fs/xfs/xfs_inode.c | 32 +++++++++++++++--- fs/xfs/xfs_iops.c | 15 ++++++++- fs/xfs/xfs_super.c | 10 ++++++ fs/xfs/xfs_xattr.c | 2 + fs/xfs/xfs_xattr.h | 2 + 11 files changed, 245 insertions(+), 13 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_trans_space.c
fs/xfs/Makefile +1 -0
@@ -51,6 +51,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_symlink_remote.o \ xfs_trans_inode.o \ xfs_trans_resv.o \ + xfs_trans_space.o \ xfs_types.o \ ) # xfs_rtbitmap is shared with libxfs
fs/xfs/libxfs/xfs_parent.c +68 -0
77 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +65 -0
69 lines not on this page · raw message
fs/xfs/libxfs/xfs_trans_space.c new file +52 -0
52 lines not on this page · raw message
fs/xfs/libxfs/xfs_trans_space.h +5 -4
@@ -80,8 +80,6 @@/* This macro is not used - see inline code in xfs_attr_set */ #define XFS_ATTRSET_SPACE_RES (mp, v) \ (XFS_DAENTER_SPACE_RES(mp, XFS_ATTR_FORK) + XFS_B_TO_FSB(mp, v)) -#define XFS_CREATE_SPACE_RES (mp,nl) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_DIOSTRAT_SPACE_RES (mp, v) \ (XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK) + (v))#define XFS_GROWFS_SPACE_RES (mp) \
@@ -90,8 +88,6 @@ ((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK))#define XFS_LINK_SPACE_RES (mp,nl) \ XFS_DIRENTER_SPACE_RES(mp,nl) -#define XFS_MKDIR_SPACE_RES (mp,nl) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_QM_DQALLOC_SPACE_RES (mp) \ (XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK) + \ XFS_DQUOT_CLUSTER_SIZE_FSB)
@@ -106,5 +102,10 @@#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0) +unsigned int xfs_parent_calc_space_res (struct xfs_mount *mp, +unsigned int namelen); + +unsigned int xfs_create_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_mkdir_space_res (struct xfs_mount *mp,unsigned int namelen);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/tempfile.c +1 -1
@@ -71,7 +71,7 @@ xrep_tempfile_create(return error;if (is_dir) { - resblks =XFS_MKDIR_SPACE_RES (mp,0 ); + resblks =xfs_mkdir_space_res (mp,0 ); tres = &M_RES (mp)->tr_mkdir; }else { resblks =XFS_IALLOC_SPACE_RES (mp);
fs/xfs/xfs_inode.c +26 -6
@@ -40,6 +40,8 @@#include "xfs_log_priv.h" #include "xfs_health.h" #include "xfs_pnfs.h" +#include "xfs_parent.h" +#include "xfs_xattr.h" struct kmem_cache *xfs_inode_cache;
@@ -1016,7 +1018,7 @@ xfs_dir_hook_setup( int xfs_create( struct mnt_idmap *idmap, - xfs_inode_t *dp, + struct xfs_inode *dp, struct xfs_name *name, umode_t mode, dev_t rdev,
@@ -1028,7 +1030,7 @@ xfs_create(struct xfs_inode *ip =NULL ;struct xfs_trans *tp =NULL ;int error; -bool unlock_dp_on_error = false; +bool unlock_dp_on_error = false;prid_t prid;struct xfs_dquot *udqp =NULL ;struct xfs_dquot *gdqp =NULL ;
@@ -1036,6 +1038,7 @@ xfs_create(struct xfs_trans_res *tres;uint resblks;xfs_ino_t ino; +struct xfs_parent_args *ppargs;trace_xfs_create (dp, name);
@@ -1057,13 +1060,17 @@ xfs_create(return error;if (is_dir) { - resblks =XFS_MKDIR_SPACE_RES (mp, name->len); + resblks =xfs_mkdir_space_res (mp, name->len); tres = &M_RES (mp)->tr_mkdir; }else { - resblks =XFS_CREATE_SPACE_RES (mp, name->len); + resblks =xfs_create_space_res (mp, name->len); tres = &M_RES (mp)->tr_create; } + error =xfs_parent_start (mp, &ppargs); +if (error) + goto out_release_dquots; +/* * Initially assume that the file does not exist and * reserve the resources for that case. If that is not
@@ -1079,7 +1086,7 @@ xfs_create( resblks, &tp); }if (error) - goto out_release_dquots; + goto out_parent;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;
@@ -1122,6 +1129,16 @@ xfs_create(xfs_bumplink (tp, dp); } +/* +* If we have parent pointers, we need to add the attribute containing +* the parent information now. +*/ +if (ppargs) { + error =xfs_parent_addname (tp, ppargs, dp, name, ip); +if (error) + goto out_trans_cancel; + } +/* * Create ip with a reference from dp, and add '.' and '..' references * if it's a directory.
@@ -1154,6 +1171,7 @@ xfs_create( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); +xfs_parent_finish (mp, ppargs);return 0 ; out_trans_cancel:
@@ -1169,6 +1187,8 @@ xfs_create(xfs_finish_inode_setup (ip);xfs_irele (ip); } + out_parent: +xfs_parent_finish (mp, ppargs); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);
@@ -3037,7 +3057,7 @@ xfs_rename_alloc_whiteout(int error; error =xfs_create_tmpfile (idmap, dp,S_IFCHR |WHITEOUT_MODE , - false, &tmpfile); +xfs_has_parent (dp->i_mount), &tmpfile);if (error)return error;
fs/xfs/xfs_iops.c +14 -1
@@ -157,6 +157,8 @@ xfs_create_need_xattr(if (dir->i_sb->s_security)return true;#endif +if (xfs_has_parent (XFS_I (dir)->i_mount)) +return true;return false; }
@@ -201,7 +203,18 @@ xfs_generic_create(xfs_create_need_xattr (dir, default_acl, acl), &ip); }else { - error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, false, &ip); +bool init_xattrs = false; + +/* +* If this temporary file will be linkable, set up the file +* with an attr fork to receive a parent pointer. +*/ +if (!(tmpfile->f_flags &O_EXCL ) && +xfs_has_parent (XFS_I (dir)->i_mount)) + init_xattrs = true; + + error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, + init_xattrs, &ip); }if (unlikely (error)) goto out_free_acl;
fs/xfs/xfs_super.c +10 -0
@@ -44,6 +44,7 @@#include "xfs_dahash_test.h" #include "xfs_rtbitmap.h" #include "xfs_exchmaps_item.h" +#include "xfs_parent.h" #include "scrub/stats.h" #include "scrub/rcbag_btree.h"
@@ -2202,8 +2203,16 @@ xfs_init_caches(void)if (!xfs_xmi_cache) goto out_destroy_xmd_cache; + xfs_parent_args_cache =kmem_cache_create ("xfs_parent_args" , +sizeof (struct xfs_parent_args ), +0 ,0 ,NULL ); +if (!xfs_parent_args_cache) + goto out_destroy_xmi_cache; +return 0 ; + out_destroy_xmi_cache: +kmem_cache_destroy (xfs_xmi_cache); out_destroy_xmd_cache:kmem_cache_destroy (xfs_xmd_cache); out_destroy_iul_cache:
@@ -2264,6 +2273,7 @@ xfs_destroy_caches(void)* destroy caches. */ rcu_barrier (); +kmem_cache_destroy (xfs_parent_args_cache);kmem_cache_destroy (xfs_xmd_cache);kmem_cache_destroy (xfs_xmi_cache);kmem_cache_destroy (xfs_iunlink_cache);
fs/xfs/xfs_xattr.c +1 -1
@@ -24,7 +24,7 @@* Get permission to use log-assisted atomic exchange of file extents. * Callers must not be running any transactions or hold any ILOCKs. */ -static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {
fs/xfs/xfs_xattr.h +2 -0
@@ -7,6 +7,8 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp); +void xfs_attr_rele_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *const xfs_xattr_handlers[];
Re: [PATCH 17/32] xfs: parent pointer attribute creation
One thing that might be worth documenting in a comment or at least the commit log is why we have this three phase split between allocating the daargs, doing all the work and freeing it.
As far as I can tell that is because the da_args need to be around until transaction commit because xfs_attr_intent has a pointer to the da_args and not a full copy. So unless the attrs are on stack they need to be free after transaction commit, and as the normal dir operation args are not on the stack we don't want to add the attr one to the stack here. We could probably allocate the da_args in the main parent pointer helpers, but that would require a NOFAIL allocation and maybe lead to odd calling conventions, but maybe someone directly involved can further refine that reasoning.
Otherwise looks good:
Re: [PATCH 17/32] xfs: parent pointer attribute creation
Quoted 8 lines · Christoph Hellwig
One thing that might be worth documenting in a comment or at least the commit log is why we have this three phase split between allocating the daargs, doing all the work and freeing it.
As far as I can tell that is because the
da_argsneed to be around until transaction commit becausexfs_attr_intenthas a pointer to theda_argsand not a full copy.
Correct. I think the xfs_attr_intent could make its own full copy and iterate on that, but that would break the existing behavior that the xfs_attr_set caller can look at the end state of the xfs_attr_set operation.
AFAICT the only xfs_attr_* callers that care about the end state at all are xfs_attr_lookup functions (because they might want to get the value). I don't *think* any of the xfs_attr_set callers actually care. The log intent creation function will snapshot the name/value/oldname/newvalue buffers, so we're already doing large(ish) allocations deep in transaction context.
On the other hand, the current code has fewer copies to make because we "know" that the da_args has to persist until transaction commit because...
Quoted 7 lines
So unless the attrs are on stack
they need to be free after transaction commit, and as the normal dir operation args are not on the stack we don't want to add the attr one to the stack here. We could probably allocate theda_argsin the main parent pointer helpers, but that would require a NOFAIL allocation and maybe lead to odd calling conventions, but maybe someone directly involved can further refine that reasoning.
...the da_args (for the parent pointer op) is already allocated as part of xfs_parent_args in xfs_parent_start. If the kernel supported alloca (HAHAAHA) then we wouldn't need the xfs_parent_args_cache.
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
PATCH 18/32 xfs: add parent attributes to link
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_link to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.c | 14 +++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 3 +-- fs/xfs/scrub/dir_repair.c | 2 +- fs/xfs/scrub/orphanage.c | 2 +- fs/xfs/xfs_inode.c | 43 ++++++++++++++++++++++++++++++++++----- 5 files changed, 54 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_trans_space.c +14 -0
@@ -50,3 +50,17 @@ xfs_mkdir_space_res( {return xfs_create_space_res (mp, namelen); } + +unsigned int +xfs_link_space_res ( +struct xfs_mount *mp, +unsigned int namelen) +{ +unsigned int ret; + + ret =XFS_DIRENTER_SPACE_RES (mp, namelen); +if (xfs_has_parent (mp)) + ret +=xfs_parent_calc_space_res (mp, namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +1 -2
@@ -86,8 +86,6 @@ (2 * (mp)->m_alloc_maxlevels)#define XFS_GROWFSRT_SPACE_RES (mp,b) \ ((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK)) -#define XFS_LINK_SPACE_RES (mp,nl) \ - XFS_DIRENTER_SPACE_RES(mp,nl)#define XFS_QM_DQALLOC_SPACE_RES (mp) \ (XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK) + \ XFS_DQUOT_CLUSTER_SIZE_FSB)
@@ -107,5 +105,6 @@ unsigned int xfs_parent_calc_space_res(struct xfs_mount *mp,unsigned int xfs_create_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_mkdir_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_link_space_res (struct xfs_mount *mp,unsigned int namelen);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/dir_repair.c +1 -1
@@ -704,7 +704,7 @@ xrep_dir_replay_update(uint resblks;int error; - resblks =XFS_LINK_SPACE_RES (mp, xname->len); + resblks =xfs_link_space_res (mp, xname->len); error =xchk_trans_alloc (rd->sc, resblks);if (error)return error;
fs/xfs/scrub/orphanage.c +1 -1
@@ -326,7 +326,7 @@ xrep_adoption_trans_alloc(/* Compute the worst case space reservation that we need. */ adopt->sc = sc; - adopt->orphanage_blkres =XFS_LINK_SPACE_RES (mp,MAXNAMELEN ); + adopt->orphanage_blkres =xfs_link_space_res (mp,MAXNAMELEN );if (S_ISDIR (VFS_I (sc->ip)->i_mode)) child_blkres =XFS_RENAME_SPACE_RES (mp, xfs_name_dotdot.len); adopt->child_blkres = child_blkres;
fs/xfs/xfs_inode.c +37 -6
80 lines not on this page · raw message
Re: [PATCH 18/32] xfs: add parent attributes to link
Looks good:
PATCH 19/32 xfs: add parent attributes to symlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_symlink to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.c | 17 +++++++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 4 ++-- fs/xfs/scrub/symlink_repair.c | 2 +- fs/xfs/xfs_symlink.c | 30 +++++++++++++++++++++++++----- 4 files changed, 45 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_trans_space.c +17 -0
@@ -64,3 +64,20 @@ xfs_link_space_res(return ret; } + +unsigned int +xfs_symlink_space_res ( +struct xfs_mount *mp, +unsigned int namelen, +unsigned int fsblocks) +{ +unsigned int ret; + + ret =XFS_IALLOC_SPACE_RES (mp) +XFS_DIRENTER_SPACE_RES (mp, namelen) + + fsblocks; + +if (xfs_has_parent (mp)) + ret +=xfs_parent_calc_space_res (mp, namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +2 -2
@@ -95,8 +95,6 @@ XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl)) -#define XFS_SYMLINK_SPACE_RES (mp,nl,b) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl) + (b))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
@@ -106,5 +104,7 @@ unsigned int xfs_parent_calc_space_res(struct xfs_mount *mp,unsigned int xfs_create_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_mkdir_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_link_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_symlink_space_res (struct xfs_mount *mp,unsigned int namelen, +unsigned int fsblocks);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/symlink_repair.c +1 -1
@@ -421,7 +421,7 @@ xrep_symlink_rebuild(* unlikely. */ fs_blocks =xfs_symlink_blocks (sc->mp, target_len); - resblks =XFS_SYMLINK_SPACE_RES (sc->mp, target_len, fs_blocks); + resblks =xfs_symlink_space_res (sc->mp, target_len, fs_blocks); error =xfs_trans_reserve_quota_nblks (sc->tp, sc->tempip, resblks,0 , true);if (error)
fs/xfs/xfs_symlink.c +25 -5
@@ -25,6 +25,8 @@#include "xfs_error.h" #include "xfs_health.h" #include "xfs_symlink_remote.h" +#include "xfs_parent.h" +#include "xfs_defer.h" int xfs_readlink(
@@ -100,6 +102,7 @@ xfs_symlink(struct xfs_dquot *pdqp =NULL ;uint resblks;xfs_ino_t ino; +struct xfs_parent_args *ppargs; *ipp =NULL ;
@@ -130,18 +133,24 @@ xfs_symlink(/* * The symlink will fit into the inode data fork? -* There can't be any attributes so we get the whole variable part. +* If there are no parent pointers, then there wont't be any attributes. +* So we get the whole variable part, and do not need to reserve extra +* blocks. Otherwise, we need to reserve the blocks. */ -if (pathlen <=XFS_LITINO (mp)) +if (pathlen <=XFS_LITINO (mp) && !xfs_has_parent (mp)) fs_blocks =0 ;else fs_blocks =xfs_symlink_blocks (mp, pathlen); - resblks =XFS_SYMLINK_SPACE_RES (mp, link_name->len, fs_blocks); + resblks =xfs_symlink_space_res (mp, link_name->len, fs_blocks); + + error =xfs_parent_start (mp, &ppargs); +if (error) + goto out_release_dquots; error =xfs_trans_alloc_icreate (mp, &M_RES (mp)->tr_symlink, udqp, gdqp, pdqp, resblks, &tp);if (error) - goto out_release_dquots; + goto out_parent;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;
@@ -161,7 +170,7 @@ xfs_symlink(if (!error) error =xfs_init_new_inode (idmap, tp, dp, ino,S_IFLNK | (mode & ~S_IFMT ),1 ,0 , prid, - false, &ip); +xfs_has_parent (mp), &ip);if (error) goto out_trans_cancel;
@@ -195,6 +204,14 @@ xfs_symlink( goto out_trans_cancel;xfs_trans_ichgtime (tp, dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, dp,XFS_ILOG_CORE ); + +/* Add parent pointer for the new symlink. */ +if (ppargs) { + error =xfs_parent_addname (tp, ppargs, dp, link_name, ip); +if (error) + goto out_trans_cancel; + } +xfs_dir_update_hook (dp, ip,1 , link_name);/*
@@ -216,6 +233,7 @@ xfs_symlink( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); +xfs_parent_finish (mp, ppargs);return 0 ; out_trans_cancel:
@@ -231,6 +249,8 @@ xfs_symlink(xfs_finish_inode_setup (ip);xfs_irele (ip); } +out_parent: +xfs_parent_finish (mp, ppargs); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);
Re: [PATCH 19/32] xfs: add parent attributes to symlink
Looks good:
PATCH 20/32 xfs: remove parent pointers in unlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the parent pointer attribute during unlink
--- fs/xfs/libxfs/xfs_parent.c | 22 ++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 3 +++ fs/xfs/libxfs/xfs_trans_space.c | 13 +++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 3 +-- fs/xfs/xfs_inode.c | 27 +++++++++++++++++++++------ 5 files changed, 60 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_parent.c +22 -0
@@ -207,3 +207,25 @@ xfs_parent_addname(xfs_attr_defer_parent (&ppargs->args,XFS_ATTR_DEFER_SET );return 0 ; } + +/* Remove a parent pointer to reflect a dirent removal. */ +int +xfs_parent_removename ( +struct xfs_trans *tp, +struct xfs_parent_args *ppargs, +struct xfs_inode *dp, +const struct xfs_name *parent_name, +struct xfs_inode *child) +{ +int error; + + error =xfs_parent_iread_extents (tp, child); +if (error) +return error; + +xfs_inode_to_parent_rec (&ppargs->rec, dp); +xfs_parent_da_args_init (&ppargs->args, tp, &ppargs->rec, child, + child->i_ino, parent_name); +xfs_attr_defer_parent (&ppargs->args,XFS_ATTR_DEFER_REMOVE ); +return 0 ; +}
fs/xfs/libxfs/xfs_parent.h +3 -0
@@ -81,5 +81,8 @@ xfs_parent_finish(int xfs_parent_addname (struct xfs_trans *tp,struct xfs_parent_args *ppargs,struct xfs_inode *dp,const struct xfs_name *parent_name,struct xfs_inode *child); +int xfs_parent_removename (struct xfs_trans *tp,struct xfs_parent_args *ppargs, +struct xfs_inode *dp,const struct xfs_name *parent_name, +struct xfs_inode *child);#endif /* __XFS_PARENT_H__ */
fs/xfs/libxfs/xfs_trans_space.c +13 -0
@@ -81,3 +81,16 @@ xfs_symlink_space_res(return ret; } + +unsigned int +xfs_remove_space_res ( +struct xfs_mount *mp, +unsigned int namelen) +{ +unsigned int ret =XFS_DIRREMOVE_SPACE_RES (mp); + +if (xfs_has_parent (mp)) + ret +=xfs_parent_calc_space_res (mp, namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +1 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_REMOVE_SPACE_RES (mp) \ - XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \
@@ -106,5 +104,6 @@ unsigned int xfs_mkdir_space_res(struct xfs_mount *mp, unsigned int namelen);unsigned int xfs_link_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_symlink_space_res (struct xfs_mount *mp,unsigned int namelen,unsigned int fsblocks); +unsigned int xfs_remove_space_res (struct xfs_mount *mp,unsigned int namelen);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/xfs_inode.c +21 -6
@@ -2720,16 +2720,17 @@ xfs_iunpin_wait(*/ int xfs_remove ( -xfs_inode_t *dp, +struct xfs_inode *dp,struct xfs_name *name, -xfs_inode_t *ip) +struct xfs_inode *ip) { -xfs_mount_t *mp = dp->i_mount; -xfs_trans_t *tp =NULL ; +struct xfs_mount *mp = dp->i_mount; +struct xfs_trans *tp =NULL ;int is_dir =S_ISDIR (VFS_I (ip)->i_mode);int dontcare;int error =0 ;uint resblks; +struct xfs_parent_args *ppargs;trace_xfs_remove (dp, name);
@@ -2746,6 +2747,10 @@ xfs_remove(if (error) goto std_return; + error =xfs_parent_start (mp, &ppargs); +if (error) + goto std_return; +/* * We try to get the real space reservation first, allowing for * directory btree deletion(s) implying possible bmap insert(s). If we
@@ -2757,12 +2762,12 @@ xfs_remove(* the directory code can handle a reservationless update and we don't * want to prevent a user from trying to free space by deleting things. */ - resblks =XFS_REMOVE_SPACE_RES (mp); + resblks =xfs_remove_space_res (mp, name->len); error =xfs_trans_alloc_dir (dp, &M_RES (mp)->tr_remove, ip, &resblks, &tp, &dontcare);if (error) {ASSERT (error != -ENOSPC ); - goto std_return; + goto out_parent; }/*
@@ -2822,6 +2827,13 @@ xfs_remove( goto out_trans_cancel; } +/* Remove parent pointer. */ +if (ppargs) { + error =xfs_parent_removename (tp, ppargs, dp, name, ip); +if (error) + goto out_trans_cancel; + } +/* * Drop the link from dp to ip, and if ip was a directory, remove the * '.' and '..' references since we freed the directory.
@@ -2845,6 +2857,7 @@ xfs_remove(xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); +xfs_parent_finish (mp, ppargs);return 0 ; out_trans_cancel:
@@ -2852,6 +2865,8 @@ xfs_remove( out_unlock:xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); + out_parent: +xfs_parent_finish (mp, ppargs); std_return:return error; }
Re: [PATCH 20/32] xfs: remove parent pointers in unlink
Looks good:
PATCH 21/32 xfs: Add parent pointers to rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer.
--- fs/xfs/libxfs/xfs_parent.c | 30 +++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 6 +++ fs/xfs/libxfs/xfs_trans_space.c | 25 ++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 6 ++- fs/xfs/scrub/orphanage.c | 3 + fs/xfs/scrub/parent_repair.c | 3 + fs/xfs/xfs_inode.c | 80 ++++++++++++++++++++++++++++++++++++--- 7 files changed, 142 insertions(+), 11 deletions(-)
fs/xfs/libxfs/xfs_parent.c +30 -0
@@ -229,3 +229,33 @@ xfs_parent_removename(xfs_attr_defer_parent (&ppargs->args,XFS_ATTR_DEFER_REMOVE );return 0 ; } + +/* Replace one parent pointer with another to reflect a rename. */ +int +xfs_parent_replacename ( +struct xfs_trans *tp, +struct xfs_parent_args *ppargs, +struct xfs_inode *old_dp, +const struct xfs_name *old_name, +struct xfs_inode *new_dp, +const struct xfs_name *new_name, +struct xfs_inode *child) +{ +int error; + + error =xfs_parent_iread_extents (tp, child); +if (error) +return error; + +xfs_inode_to_parent_rec (&ppargs->rec, old_dp); +xfs_parent_da_args_init (&ppargs->args, tp, &ppargs->rec, child, + child->i_ino, old_name); + +xfs_inode_to_parent_rec (&ppargs->new_rec, new_dp); + ppargs->args.new_name = new_name->name; + ppargs->args.new_namelen = new_name->len; + ppargs->args.new_value = &ppargs->new_rec; + ppargs->args.new_valuelen =sizeof (struct xfs_parent_rec ); +xfs_attr_defer_parent (&ppargs->args,XFS_ATTR_DEFER_REPLACE ); +return 0 ; +}
fs/xfs/libxfs/xfs_parent.h +6 -0
@@ -45,6 +45,7 @@ extern struct kmem_cache *xfs_parent_args_cache;*/ struct xfs_parent_args {struct xfs_parent_rec rec; +struct xfs_parent_rec new_rec;struct xfs_da_args args; };
@@ -84,5 +85,10 @@ int xfs_parent_addname(struct xfs_trans *tp, struct xfs_parent_args *ppargs,int xfs_parent_removename (struct xfs_trans *tp,struct xfs_parent_args *ppargs,struct xfs_inode *dp,const struct xfs_name *parent_name,struct xfs_inode *child); +int xfs_parent_replacename (struct xfs_trans *tp, +struct xfs_parent_args *ppargs, +struct xfs_inode *old_dp,const struct xfs_name *old_name, +struct xfs_inode *new_dp,const struct xfs_name *new_name, +struct xfs_inode *child);#endif /* __XFS_PARENT_H__ */
fs/xfs/libxfs/xfs_trans_space.c +25 -0
@@ -94,3 +94,28 @@ xfs_remove_space_res(return ret; } + +unsigned int +xfs_rename_space_res ( +struct xfs_mount *mp, +unsigned int src_namelen, +bool target_exists, +unsigned int target_namelen, +bool has_whiteout) +{ +unsigned int ret; + + ret =XFS_DIRREMOVE_SPACE_RES (mp) + +XFS_DIRENTER_SPACE_RES (mp, target_namelen); + +if (xfs_has_parent (mp)) { +if (has_whiteout) + ret +=xfs_parent_calc_space_res (mp, src_namelen); + ret +=2 *xfs_parent_calc_space_res (mp, target_namelen); + } + +if (target_exists) + ret +=xfs_parent_calc_space_res (mp, target_namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +4 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_RENAME_SPACE_RES (mp,nl) \ - (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
@@ -106,4 +104,8 @@ unsigned int xfs_symlink_space_res(struct xfs_mount *mp, unsigned int namelen, unsigned int fsblocks);unsigned int xfs_remove_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_rename_space_res (struct xfs_mount *mp, +unsigned int src_namelen,bool target_exists, +unsigned int target_namelen,bool has_whiteout); +#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/orphanage.c +2 -1
@@ -328,7 +328,8 @@ xrep_adoption_trans_alloc( adopt->sc = sc; adopt->orphanage_blkres =xfs_link_space_res (mp,MAXNAMELEN );if (S_ISDIR (VFS_I (sc->ip)->i_mode)) - child_blkres =XFS_RENAME_SPACE_RES (mp, xfs_name_dotdot.len); + child_blkres =xfs_rename_space_res (mp,0 , false, + xfs_name_dotdot.len, false); adopt->child_blkres = child_blkres;/*
fs/xfs/scrub/parent_repair.c +2 -1
@@ -171,7 +171,8 @@ xrep_parent_reset_dotdot(* Reserve more space just in case we have to expand the dir. We're * allowed to exceed quota to repair inconsistent metadata. */ - spaceres =XFS_RENAME_SPACE_RES (sc->mp, xfs_name_dotdot.len); + spaceres =xfs_rename_space_res (sc->mp,0 , false, xfs_name_dotdot.len, + false); error =xfs_trans_reserve_more_inode (sc->tp, sc->ip, spaceres,0 , true);if (error)
fs/xfs/xfs_inode.c +73 -7
128 lines not on this page · raw message
Re: [PATCH 21/32] xfs: Add parent pointers to rename
Looks good:
PATCH 22/32 xfs: Add parent pointers to xfs_cross_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Cross renames are handled separately from standard renames, and need different handling to update the parent attributes correctly.
--- fs/xfs/xfs_inode.c | 33 +++++++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 8 deletions(-)
fs/xfs/xfs_inode.c +25 -8
@@ -2971,15 +2971,17 @@ xfs_cross_rename(struct xfs_inode *dp1, struct xfs_name *name1, struct xfs_inode *ip1, + struct xfs_parent_args *ip1_ppargs, struct xfs_inode *dp2, struct xfs_name *name2, struct xfs_inode *ip2, + struct xfs_parent_args *ip2_ppargs, int spaceres) { - int error = 0; -int ip1_flags =0 ; -int ip2_flags =0 ; -int dp2_flags =0 ; + int error = 0; +int ip1_flags =0 ; +int ip2_flags =0 ; +int dp2_flags =0 ;/* Swap inode number for dirent in first parent */ error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres);
@@ -3048,6 +3050,21 @@ xfs_cross_rename( } } +/* Schedule parent pointer replacements */ +if (ip1_ppargs) { + error =xfs_parent_replacename (tp, ip1_ppargs, dp1, name1, dp2, + name2, ip1); +if (error) + goto out_trans_abort; + } + +if (ip2_ppargs) { + error =xfs_parent_replacename (tp, ip2_ppargs, dp2, name2, dp1, + name1, ip2); +if (error) + goto out_trans_abort; + } +if (ip1_flags) {xfs_trans_ichgtime (tp, ip1, ip1_flags);xfs_trans_log_inode (tp, ip1,XFS_ILOG_CORE );
@@ -3264,10 +3281,10 @@ xfs_rename(/* RENAME_EXCHANGE is unique from here on. */ if (flags &RENAME_EXCHANGE ) { error =xfs_cross_rename (tp, src_dp, src_name, src_ip, - target_dp, target_name, target_ip, - spaceres); -xfs_iunlock_rename (inodes, num_inodes); -return error; + src_ppargs, target_dp, target_name, target_ip, + tgt_ppargs, spaceres); + nospace_error =0 ; + goto out_unlock; }/*
Re: [PATCH 22/32] xfs: Add parent pointers to xfs_cross_rename
Looks good:
PATCH 23/32 xfs: Filter XFS_ATTR_PARENT for getfattr
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Parent pointers returned to the get_fattr tool cause errors since the tool cannot parse parent pointers. Fix this by filtering parent parent pointers from xfs_xattr_put_listent.
--- fs/xfs/libxfs/xfs_da_format.h | 3 +++ fs/xfs/xfs_xattr.c | 10 ++++++++++ 2 files changed, 13 insertions(+)
fs/xfs/libxfs/xfs_da_format.h +3 -0
@@ -726,6 +726,9 @@ struct xfs_attr3_leafblock { XFS_ATTR_SECURE | \ XFS_ATTR_PARENT) +/* Private attr namespaces not exposed to userspace */ +#define XFS_ATTR_PRIVATE_NSP_MASK (XFS_ATTR_PARENT) +#define XFS_ATTR_ONDISK_MASK (XFS_ATTR_NSP_ONDISK_MASK | \ XFS_ATTR_LOCAL | \ XFS_ATTR_INCOMPLETE)
fs/xfs/xfs_xattr.c +10 -0
@@ -20,6 +20,12 @@#include <linux/posix_acl_xattr.h> +/* +* This file defines functions to work with externally visible extended +* attributes, such as those in user, system, or security namespaces. They +* should not be used for internally used attributes. Consider xfs_attr.c. +*/ +/* * Get permission to use log-assisted atomic exchange of file extents. * Callers must not be running any transactions or hold any ILOCKs.
@@ -215,6 +221,10 @@ xfs_xattr_put_listent(ASSERT (context->count >=0 ); +/* Don't expose private xattr namespaces. */ +if (flags &XFS_ATTR_PRIVATE_NSP_MASK ) +return ; +if (flags &XFS_ATTR_ROOT ) {#ifdef CONFIG_XFS_POSIX_ACL if (namelen == SGI_ACL_FILE_SIZE &&
Re: [PATCH 23/32] xfs: Filter XFS_ATTR_PARENT for getfattr
Quoted 5 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
Parent pointers returned to the
get_fattrtool cause errors since the tool cannot parse parent pointers. Fix this by filtering parent parent pointers fromxfs_xattr_put_listent.
With the new format returning the attrs should not cause parsing errors. OTOH we now have duplicate names, which means a get operation based on the name can't actually work in that case.
I'd also argue that parent pointers are internal enough that they should not be exposed through the normal xattr interfaces.
Quoted 5 lines
+/*
+ * This file defines functions to work with externally visible extended
+ * attributes, such as those in user, system, or security namespaces. They
+ * should not be used for internally used attributes. Considerxfs_attr.c.
+ */
As long as xfs_attr_change and xfs_attr_grab_log_assist are xfs_xattr.c that is not actually true. However I think they should be moved to xfs_attr.c (and in case of xfs_attr_change merged into xfs_attr_set) to make this comment true.
However I'd make it part of the top of file comment above the include statements. And please add it in a separate commit as it has nothing to do with the other changes here.
Re: [PATCH 23/32] xfs: Filter XFS_ATTR_PARENT for getfattr
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
Parent pointers returned to the
get_fattrtool cause errors since the tool cannot parse parent pointers. Fix this by filtering parent parent pointers fromxfs_xattr_put_listent.
Quoted 6 lines
With the new format returning the attrs should not cause parsing errors. OTOH we now have duplicate names, which means a get operation based on the name can't actually work in that case.
I'd also argue that parent pointers are internal enough that they should not be exposed through the normal xattr interfaces.
Yeah, I probably should change the commit message to:
"xfs: don't return XFS_ATTR_PARENT attributes via listxattr
"Parent pointers are internal filesystem metadata. They're not intended to be directly visible to userspace, so filter them out of xfs_xattr_put_listent so that they don't appear in listxattr."
Quoted 10 lines
Quoted 5 lines
+/*
+ * This file defines functions to work with externally visible extended
+ * attributes, such as those in user, system, or security namespaces. They
+ * should not be used for internally used attributes. Considerxfs_attr.c.
+ */As long as
xfs_attr_changeandxfs_attr_grab_log_assistarexfs_xattr.cthat is not actually true. However I think they should be moved toxfs_attr.c(and in case ofxfs_attr_changemerged intoxfs_attr_set) to make this comment true.
I don't want to hoist all the larp enabling jun^Wmachinery to libxfs and then have to stub that out in userspace. I'd rather get rid of larp mode entirely, after which point xfs_attr_change becomes a trivial helper that can be collapsed.
Quoted 3 lines
However I'd make it part of the top of file comment above the include statements. And please add it in a separate commit as it has nothing to do with the other changes here.
Or just get rid of the comment entirely? It came from the verity series.
--D
Re: [PATCH 23/32] xfs: Filter XFS_ATTR_PARENT for getfattr
Quoted 5 lines · Darrick J. Wong
"xfs: don't return
XFS_ATTR_PARENTattributes via listxattr"Parent pointers are internal filesystem metadata. They're not intended to be directly visible to userspace, so filter them out of
xfs_xattr_put_listentso that they don't appear in listxattr."
Looks good.
Quoted 6 lines
Quoted 3 lines
However I'd make it part of the top of file comment above the include statements. And please add it in a separate commit as it has nothing to do with the other changes here.
Or just get rid of the comment entirely? It came from the verity series.
Fine with me.
PATCH 24/32 xfs: pass the attr value to put_listent when possible
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Pass the attr value to put_listent when we have local xattrs or shortform xattrs.
--- fs/xfs/libxfs/xfs_attr.h | 5 +++-- fs/xfs/libxfs/xfs_attr_sf.h | 1 + fs/xfs/xfs_attr_list.c | 8 +++++++- fs/xfs/xfs_ioctl.c | 1 + fs/xfs/xfs_xattr.c | 1 + 5 files changed, 13 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_attr.h +3 -2
@@ -47,8 +47,9 @@ struct xfs_attrlist_cursor_kern {/* void; state communicated via *context */ -typedef void (*put_listent_func_t )(struct xfs_attr_list_context *,int , -unsigned char *,int ,int ); +typedef void (*put_listent_func_t )(struct xfs_attr_list_context *context, +int flags,unsigned char *name,int namelen,void *value, +int valuelen);struct xfs_attr_list_context {struct xfs_trans *tp;
fs/xfs/libxfs/xfs_attr_sf.h +1 -0
@@ -16,6 +16,7 @@ typedef struct xfs_attr_sf_sort {uint8_t flags;/* flags bits (see xfs_attr_leaf.h) */ xfs_dahash_t hash;/* this entry's hash value */ unsigned char *name;/* name value, pointer into buffer */ +void *value; } xfs_attr_sf_sort_t;#define XFS_ATTR_SF_ENTSIZE_MAX /* max space for name&value */ \
fs/xfs/xfs_attr_list.c +7 -1
@@ -92,6 +92,7 @@ xfs_attr_shortform_list( sfe->flags, sfe->nameval, (int )sfe->namelen, + &sfe->nameval[sfe->namelen], (int)sfe->valuelen);/* * Either search callback finished early or
@@ -138,6 +139,7 @@ xfs_attr_shortform_list( sbp->name = sfe->nameval; sbp->namelen = sfe->namelen;/* These are bytes, and both on-disk, don't endian-flip */ + sbp->value = &sfe->nameval[sfe->namelen], sbp->valuelen = sfe->valuelen; sbp->flags = sfe->flags; sbp->hash = xfs_attr_hashval(dp->i_mount, sfe->flags,
@@ -192,6 +194,7 @@ xfs_attr_shortform_list( sbp->flags, sbp->name, sbp->namelen, + sbp->value, sbp->valuelen);if (context->seen_enough)break ;
@@ -479,6 +482,7 @@ xfs_attr3_leaf_list_int(*/ for (; i < ichdr.count; entry++, i++) {char *name; +void *value;int namelen, valuelen;if (be32_to_cpu (entry->hashval) != cursor->hashval) {
@@ -496,6 +500,7 @@ xfs_attr3_leaf_list_int( name_loc =xfs_attr3_leaf_name_local (leaf, i); name = name_loc->nameval; namelen = name_loc->namelen; + value = &name_loc->nameval[name_loc->namelen]; valuelen =be16_to_cpu (name_loc->valuelen); }else {xfs_attr_leaf_name_remote_t *name_rmt;
@@ -503,6 +508,7 @@ xfs_attr3_leaf_list_int( name_rmt =xfs_attr3_leaf_name_remote (leaf, i); name = name_rmt->name; namelen = name_rmt->namelen; + value =NULL ; valuelen =be32_to_cpu (name_rmt->valuelen); }
@@ -513,7 +519,7 @@ xfs_attr3_leaf_list_int(return -EFSCORRUPTED ; } context->put_listent (context, entry->flags, - name, namelen, valuelen); + name, namelen, value, valuelen);if (context->seen_enough)break ; cursor->offset++;
fs/xfs/xfs_ioctl.c +1 -0
@@ -310,6 +310,7 @@ xfs_ioc_attr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { struct xfs_attrlist *alist = context->buffer;
fs/xfs/xfs_xattr.c +1 -0
@@ -214,6 +214,7 @@ xfs_xattr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { char *prefix;
Re: [PATCH 24/32] xfs: pass the attr value to put_listent when possible
Looks good:
PATCH 25/32 xfs: move handle ioctl code to xfs_handle.c
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Move the handle managemnet code (and the attrmulti code that uses it) to xfs_handle.c.
--- fs/xfs/Makefile | 1 fs/xfs/xfs_handle.c | 617 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_handle.h | 28 ++ fs/xfs/xfs_ioctl.c | 591 ------------------------------------------------ fs/xfs/xfs_ioctl.h | 28 -- fs/xfs/xfs_ioctl32.c | 1 6 files changed, 648 insertions(+), 618 deletions(-) create mode 100644 fs/xfs/xfs_handle.c create mode 100644 fs/xfs/xfs_handle.h
fs/xfs/Makefile +1 -0
@@ -78,6 +78,7 @@ xfs-y += xfs_aops.o \ xfs_fsmap.o \ xfs_fsops.o \ xfs_globals.o \ + xfs_handle.o \ xfs_health.o \ xfs_icache.o \ xfs_ioctl.o \
fs/xfs/xfs_handle.c new file +617 -0
617 lines not on this page · raw message
fs/xfs/xfs_handle.h new file +28 -0
@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2000-2005 Silicon Graphics, Inc. +* All rights reserved. +*/ +#ifndef __XFS_HANDLE_H__ +#define __XFS_HANDLE_H__ + +int xfs_attrlist_by_handle (struct file *parfilp, +struct xfs_fsop_attrlist_handlereq __user *p); +int xfs_attrmulti_by_handle (struct file *parfilp,void __user *arg); + +int xfs_find_handle (unsigned int cmd,struct xfs_fsop_handlereq *hreq); +int xfs_open_by_handle (struct file *parfilp,struct xfs_fsop_handlereq *hreq); +int xfs_readlink_by_handle (struct file *parfilp, +struct xfs_fsop_handlereq *hreq); + +int xfs_ioc_attrmulti_one (struct file *parfilp,struct inode *inode, +uint32_t opcode,void __user *uname,void __user *value, +uint32_t *len,uint32_t flags); +int xfs_ioc_attr_list (struct xfs_inode *dp,void __user *ubuf, +size_t bufsize,int flags, +struct xfs_attrlist_cursor __user *ucursor); + +struct dentry *xfs_handle_to_dentry (struct file *parfilp,void __user *uhandle, +u32 hlen); + +#endif /* __XFS_HANDLE_H__ */
fs/xfs/xfs_ioctl.c +1 -590
613 lines not on this page · raw message
fs/xfs/xfs_ioctl.h +0 -28
@@ -14,34 +14,6 @@ intxfs_ioc_swapext ( xfs_swapext_t *sxp); -extern int -xfs_find_handle ( -unsigned int cmd, -xfs_fsop_handlereq_t *hreq); - -extern int -xfs_open_by_handle ( -struct file *parfilp, -xfs_fsop_handlereq_t *hreq); - -extern int -xfs_readlink_by_handle ( -struct file *parfilp, -xfs_fsop_handlereq_t *hreq); - -int xfs_ioc_attrmulti_one (struct file *parfilp,struct inode *inode, -uint32_t opcode,void __user *uname,void __user *value, -uint32_t *len,uint32_t flags); -int xfs_ioc_attr_list (struct xfs_inode *dp,void __user *ubuf, -size_t bufsize,int flags, -struct xfs_attrlist_cursor __user *ucursor); - -extern struct dentry * -xfs_handle_to_dentry ( -struct file *parfilp, -void __user *uhandle, -u32 hlen); - extern int xfs_fileattr_get( struct dentry *dentry,
fs/xfs/xfs_ioctl32.c +1 -0
@@ -24,6 +24,7 @@#include "xfs_ioctl32.h" #include "xfs_trace.h" #include "xfs_sb.h" +#include "xfs_handle.h" #define _NATIVE_IOC (cmd, type) \ _IOC(_IOC_DIR(cmd), _IOC_TYPE(cmd), _IOC_NR(cmd), sizeof(type))
Re: [PATCH 25/32] xfs: move handle ioctl code to xfs_handle.c
Looks good:
PATCH 26/32 xfs: split out handle management helpers a bit
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Split out the functions that generate file/fs handles and map them back into dentries in preparation for the GETPARENTS ioctl next.
--- fs/xfs/libxfs/xfs_fs.h | 4 ++ fs/xfs/xfs_handle.c | 92 ++++++++++++++++++++++++++++++++---------------- 2 files changed, 64 insertions(+), 32 deletions(-)
fs/xfs/libxfs/xfs_fs.h +3 -1
@@ -633,7 +633,9 @@ typedef struct xfs_fsop_attrmulti_handlereq {/* * per machine unique filesystem identifier types. */ -typedef struct {__u32 val[2 ]; }xfs_fsid_t ;/* file system id type */ +typedef struct xfs_fsid { +__u32 val[2 ];/* file system id type */ +}xfs_fsid_t ; typedef struct xfs_fid { __u16 fid_len; /* length of remainder */
fs/xfs/xfs_handle.c +61 -31
122 lines not on this page · raw message
Re: [PATCH 26/32] xfs: split out handle management helpers a bit
fs/xfs/xfs_handle.c line 43
Quoted 2 lines
fs/xfs/xfs_handle.c +2
-0+ handle->ha_fid.fid_len =sizeof (struct xfs_fid ) - +sizeof (handle->ha_fid.fid_len);
If we clean this up anyway, maybe add a helper for the above calculation and share it with xfs_khandle_to_dentry?
Otherwise looks good:
Re: [PATCH 26/32] xfs: split out handle management helpers a bit
Quoted 5 lines · Christoph Hellwig
fs/xfs/xfs_handle.c line 43
Quoted 2 lines
fs/xfs/xfs_handle.c +2
-0+ handle->ha_fid.fid_len =sizeof (struct xfs_fid ) - +sizeof (handle->ha_fid.fid_len);If we clean this up anyway, maybe add a helper for the above calculation and share it with
xfs_khandle_to_dentry?
Done:
static inline size_txfs_filehandle_fid_len(void)
{
struct xfs_handle *handle = NULL;
return sizeof(struct xfs_fid) - sizeof(handle->ha_fid.fid_len);
}static inline size_t
xfs_filehandle_init(
struct xfs_mount *mp,
xfs_ino_t ino,
uint32_t gen,
struct xfs_handle *handle)
{
memcpy(&handle->ha_fsid, mp->m_fixedfsid, sizeof(struct xfs_fsid));
handle->ha_fid.fid_len = xfs_filehandle_fid_len();
...
}Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
PATCH 27/32 xfs: Add parent pointer ioctls
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
This patch adds a pair of new file ioctls to retrieve the parent pointer of a given inode. They both return the same results, but one operates on the file descriptor passed to ioctl() whereas the other allows the caller to specify a file handle for which the caller wants results.
--- fs/xfs/libxfs/xfs_fs.h | 73 ++++++++++++ fs/xfs/libxfs/xfs_ondisk.h | 5 + fs/xfs/libxfs/xfs_parent.c | 35 ++++++ fs/xfs/libxfs/xfs_parent.h | 5 + fs/xfs/xfs_handle.c | 259 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_handle.h | 5 + fs/xfs/xfs_ioctl.c | 6 + fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 92 ++++++++++++++++ 9 files changed, 480 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_fs.h +73 -0
@@ -840,6 +840,77 @@ struct xfs_commit_range { XFS_EXCHANGE_RANGE_DRY_RUN | \ XFS_EXCHANGE_RANGE_FILE1_WRITTEN) +/* Iterating parent pointers of files. */ + +/* target was the root directory */ +#define XFS_GETPARENTS_OFLAG_ROOT (1U << 0) + +/* Cursor is done iterating pptrs */ +#define XFS_GETPARENTS_OFLAG_DONE (1U << 1) + +#define XFS_GETPARENTS_OFLAGS_ALL (XFS_GETPARENTS_OFLAG_ROOT | \ + XFS_GETPARENTS_OFLAG_DONE) + +#define XFS_GETPARENTS_IFLAGS_ALL (0) + +struct xfs_getparents_rec { +struct xfs_handle gpr_parent;/* Handle to parent */ +__u16 gpr_reclen;/* Length of entire record */ +char gpr_name[];/* Null-terminated filename */ +} __packed; + +/* Iterate through this file's directory parent pointers */ +struct xfs_getparents { +/* +* Structure to track progress in iterating the parent pointers. +* Must be initialized to zeroes before the first ioctl call, and +* not touched by callers after that. +*/ +struct xfs_attrlist_cursor gp_cursor; + +/* Input flags: XFS_GETPARENTS_IFLAG* */ +__u16 gp_iflags; + +/* Output flags: XFS_GETPARENTS_OFLAG* */ +__u16 gp_oflags; + +/* Size of the gp_buffer in bytes */ +__u32 gp_bufsize; + +/* Must be set to zero */ +__u64 __pad; + +/* Pointer to a buffer in which to place xfs_getparents_rec */ +__u64 gp_buffer; +}; + +static inline struct xfs_getparents_rec * +xfs_getparents_first_rec (struct xfs_getparents *gp) +{ +return (struct xfs_getparents_rec *)(uintptr_t )gp->gp_buffer; +} + +static inline struct xfs_getparents_rec * +xfs_getparents_next_rec (struct xfs_getparents *gp, +struct xfs_getparents_rec *gpr) +{ +char *next = ((char *)gpr + gpr->gpr_reclen); +char *end = (char *)(uintptr_t )(gp->gp_buffer + gp->gp_bufsize); + +if (next >= end) +return NULL ; + +return (struct xfs_getparents_rec *)next; +} + +/* Iterate through this file handle's directory parent pointers. */ +struct xfs_getparents_by_handle { +/* Handle to file whose parents we want. */ +struct xfs_handle gph_handle; + +struct xfs_getparents gph_request; +}; +/* * ioctl commands that are used by Linux filesystems */
@@ -875,6 +946,8 @@ struct xfs_commit_range {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +#define XFS_IOC_GETPARENTS _IOWR('X', 62, struct xfs_getparents) +#define XFS_IOC_GETPARENTS_BY_HANDLE _IOWR('X', 63, struct xfs_getparents_by_handle)/* * ioctl commands that replace IRIX syssgi()'s
fs/xfs/libxfs/xfs_ondisk.h +5 -0
@@ -156,6 +156,11 @@ xfs_check_ondisk_structs(void) XFS_CHECK_OFFSET(struct xfs_efi_log_format_32, efi_extents, 16); XFS_CHECK_OFFSET(struct xfs_efi_log_format_64, efi_extents, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents_rec, 26); + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents, 40); + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents_by_handle, 64); +/* * The v5 superblock format extended several v4 header structures with * additional data. While new fields are only accessible on v5
fs/xfs/libxfs/xfs_parent.c +35 -0
@@ -259,3 +259,38 @@ xfs_parent_replacename(xfs_attr_defer_parent (&ppargs->args,XFS_ATTR_DEFER_REPLACE );return 0 ; } + +/* +* Extract parent pointer information from any xattr into @parent_ino/gen. +* The last two parameters can be NULL pointers. +* +* Returns 1 if this is a valid parent pointer; 0 if this is not a parent +* pointer xattr at all; or -EFSCORRUPTED for garbage. +*/ +int +xfs_parent_from_xattr ( +struct xfs_mount *mp, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +xfs_ino_t *parent_ino, +uint32_t *parent_gen) +{ +const struct xfs_parent_rec *rec = value; + +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ; + +if (!xfs_parent_namecheck (attr_flags, name, namelen)) +return -EFSCORRUPTED ; +if (!xfs_parent_valuecheck (mp, value, valuelen)) +return -EFSCORRUPTED ; + +if (parent_ino) + *parent_ino =be64_to_cpu (rec->p_ino); +if (parent_gen) + *parent_gen =be32_to_cpu (rec->p_gen); +return 1 ; +}
fs/xfs/libxfs/xfs_parent.h +5 -0
@@ -91,4 +91,9 @@ int xfs_parent_replacename(struct xfs_trans *tp, struct xfs_inode *new_dp, const struct xfs_name *new_name, struct xfs_inode *child); +int xfs_parent_from_xattr (struct xfs_mount *mp,unsigned int attr_flags, +const unsigned char *name,unsigned int namelen, +const void *value,unsigned int valuelen, +xfs_ino_t *parent_ino,uint32_t *parent_gen); +#endif /* __XFS_PARENT_H__ */
fs/xfs/xfs_handle.c +259 -0
268 lines not on this page · raw message
fs/xfs/xfs_handle.h +5 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* * Copyright (c) 2000-2005 Silicon Graphics, Inc. +* Copyright (c) 2022-2024 Oracle. * All rights reserved. */ #ifndef __XFS_HANDLE_H__
@@ -25,4 +26,8 @@ int xfs_ioc_attr_list(struct xfs_inode *dp, void __user *ubuf,struct dentry *xfs_handle_to_dentry (struct file *parfilp,void __user *uhandle,u32 hlen); +int xfs_ioc_getparents (struct file *file,struct xfs_getparents __user *arg); +int xfs_ioc_getparents_by_handle (struct file *file, +struct xfs_getparents_by_handle __user *arg); +#endif /* __XFS_HANDLE_H__ */
fs/xfs/xfs_ioctl.c +5 -1
@@ -35,6 +35,7 @@#include "xfs_health.h" #include "xfs_reflink.h" #include "xfs_ioctl.h" +#include "xfs_xattr.h" #include "xfs_rtbitmap.h" #include "xfs_file.h" #include "xfs_exchrange.h"
@@ -1542,7 +1543,10 @@ xfs_file_ioctl(case XFS_IOC_FSGETXATTRA :return xfs_ioc_fsgetxattra (ip, arg); - +case XFS_IOC_GETPARENTS : +return xfs_ioc_getparents (filp, arg); +case XFS_IOC_GETPARENTS_BY_HANDLE : +return xfs_ioc_getparents_by_handle (filp, arg);case XFS_IOC_GETBMAP :case XFS_IOC_GETBMAPA :case XFS_IOC_GETBMAPX :
fs/xfs/xfs_trace.c +1 -0
@@ -41,6 +41,7 @@#include "xfs_bmap.h" #include "xfs_exchmaps.h" #include "xfs_exchrange.h" +#include "xfs_parent.h" /* * We include this last to have the helpers above available for the trace
fs/xfs/xfs_trace.h +92 -0
104 lines not on this page · raw message
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Maybe replace the subject with 'add parent pointer listing ioctls' ?
Quoted 11 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This patch adds a pair of new file ioctls to retrieve the parent pointer of a given inode. They both return the same results, but one operates on the file descriptor passed to
ioctl()whereas the other allows the caller to specify a file handle for which the caller wants results.Signed-off-by Allison Henderson <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>[djwong: adjust to new ondisk format, split ioctls]Signed-off-by Darrick J. Wong <[email protected]>
Note that the first signoff should always be from the patch author. as recorded in the From line.
fs/xfs/libxfs/xfs_fs.h struct xfs_commit_range { · line 881
Quoted 5 lines
fs/xfs/libxfs/xfs_fs.h +5
-0+/* Size of the gp_buffer in bytes */ +__u32 gp_bufsize; + +/* Must be set to zero */ +__u64 __pad;
We don't really need this as padding. If you want to keep it for extensibility (although I can't really think of anything to use it for in the future) it should probably be renamed to gp_reserved;
fs/xfs/libxfs/xfs_fs.h struct xfs_commit_range { · line 903
Quoted 11 lines
fs/xfs/libxfs/xfs_fs.h +11
-0+static inline struct xfs_getparents_rec * +xfs_getparents_next_rec (struct xfs_getparents *gp, +struct xfs_getparents_rec *gpr) +{ +char *next = ((char *)gpr + gpr->gpr_reclen); +char *end = (char *)(uintptr_t )(gp->gp_buffer + gp->gp_bufsize); + +if (next >= end) +return NULL ; + +return (struct xfs_getparents_rec *)next;
We rely on void pointer arithmetics everywhere in the kernel and xfsprogs, so maybe use that here and avoid the need for the cast at the end?
Quoted 6 lines
+ */
fs/xfs/libxfs/xfs_parent.c +5
-0+int +xfs_parent_from_xattr( + struct xfs_mount *mp, + unsigned int attr_flags, + const unsigned char *name,
fs/xfs/libxfs/xfs_parent.c xfs_parent_replacename( · line 284
Quoted 10 lines
fs/xfs/libxfs/xfs_parent.c +10
-0+unsigned int namelen, + const void *value, + unsigned int valuelen, + xfs_ino_t *parent_ino, + uint32_t *parent_gen) +{ + const struct xfs_parent_rec *rec = value; + +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ;
I wonder if this check should move to the callers. That makes the calling conventions a lot simpler, and I think it probably makes the code a bit easier to follow as well. But I'm not entirely sure either and open for arguments.
fs/xfs/xfs_handle.c xfs_attrmulti_by_handle( · line 672
Quoted 7 lines
fs/xfs/xfs_handle.c +7
-0+static inline unsigned int +xfs_getparents_rec_sizeof ( +unsigned int namelen) +{ +return round_up (sizeof (struct xfs_getparents_rec ) + namelen +1 , +sizeof (uint32_t )); +}
As we marked the xfs_getparents_rec as __packed we shouldn't really need the alignment here. Or if we align, it should be to 8 bytes, in which case we don't need to pack it.
fs/xfs/xfs_handle.c xfs_attrmulti_by_handle( · line 689
Quoted 1 line
fs/xfs/xfs_handle.c +1
-0+unsigned short reclen =xfs_getparents_rec_sizeof (namelen);
Please avoid the overly long line here.
Otherwise looks good:
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 10 lines · Christoph Hellwig
Maybe replace the subject with 'add parent pointer listing ioctls' ?
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This patch adds a pair of new file ioctls to retrieve the parent pointer of a given inode. They both return the same results, but one operates on the file descriptor passed to
ioctl()whereas the other allows the caller to specify a file handle for which the caller wants results.
Quoted 7 lines
Quoted 4 lines
Signed-off-by Allison Henderson <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>[djwong: adjust to new ondisk format, split ioctls]Signed-off-by Darrick J. Wong <[email protected]>Note that the first signoff should always be from the patch author. as recorded in the From line.
Yeah. At this point the ioctl is so much different from Allison's original version that it doesn't make much sense to keep her as the patch author or sob person.
Quoted 9 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_commit_range { · line 881
Quoted 5 lines
fs/xfs/libxfs/xfs_fs.h +5
-0+/* Size of the gp_buffer in bytes */ +__u32 gp_bufsize; + +/* Must be set to zero */ +__u64 __pad;We don't really need this as padding. If you want to keep it for extensibility (although I can't really think of anything to use it for in the future) it should probably be renamed to
gp_reserved;
Eh, I'll keep it, just in case. The getparents_by_handle aligns nicely with a single cacheline. :P
Quoted 7 lines
Quoted 7 lines
fs/xfs/libxfs/xfs_fs.h +7
-0+static inline struct xfs_getparents_rec * +xfs_getparents_next_rec (struct xfs_getparents *gp, +struct xfs_getparents_rec *gpr) +{ +char *next = ((char *)gpr + gpr->gpr_reclen); +char *end = (char *)(uintptr_t )(gp->gp_buffer + gp->gp_bufsize); +
Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_commit_range { · line 903
Quoted 4 lines
fs/xfs/libxfs/xfs_fs.h +4
-0+if (next >= end) +return NULL ; + +return (struct xfs_getparents_rec *)next;We rely on void pointer arithmetics everywhere in the kernel and xfsprogs, so maybe use that here and avoid the need for the cast at the end?
Hopefully our downstream users also have compilers that allow void pointer arithmetic. ;)
Quoted 13 lines
Quoted 13 lines
+ */
fs/xfs/libxfs/xfs_parent.c +12
-0+int +xfs_parent_from_xattr ( +struct xfs_mount *mp, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +xfs_ino_t *parent_ino, +uint32_t *parent_gen) +{ +const struct xfs_parent_rec *rec = value;
Quoted 8 lines
fs/xfs/libxfs/xfs_parent.c xfs_parent_replacename( · line 284
Quoted 3 lines
fs/xfs/libxfs/xfs_parent.c +3
-0+ +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ;I wonder if this check should move to the callers. That makes the calling conventions a lot simpler, and I think it probably makes the code a bit easier to follow as well. But I'm not entirely sure either and open for arguments.
Yeah, on further thought I don't like the 0/1 return value convention and will change that to require callers to screen for ATTR_PARENT.
Quoted 11 lines
fs/xfs/xfs_handle.c xfs_attrmulti_by_handle( · line 672
Quoted 7 lines
fs/xfs/xfs_handle.c +7
-0+static inline unsigned int +xfs_getparents_rec_sizeof ( +unsigned int namelen) +{ +return round_up (sizeof (struct xfs_getparents_rec ) + namelen +1 , +sizeof (uint32_t )); +}As we marked the
xfs_getparents_recas__packedwe shouldn't really need the alignment here. Or if we align, it should be to 8 bytes, in which case we don't need to pack it.
Let's align it to u64; everything else is.
Quoted 3 lines
fs/xfs/xfs_handle.c xfs_attrmulti_by_handle( · line 689
Quoted 1 line
fs/xfs/xfs_handle.c +1
-0+unsigned short reclen =xfs_getparents_rec_sizeof (namelen);Please avoid the overly long line here.
Fixed.
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
In reply to Darrick J. WongQuoted 438 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This patch adds a pair of new file ioctls to retrieve the parent pointer of a given inode. They both return the same results, but one operates on the file descriptor passed to
ioctl()whereas the other allows the caller to specify a file handle for which the caller wants results.Signed-off-by Allison Henderson <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>[djwong: adjust to new ondisk format, split ioctls]Signed-off-by Darrick J. Wong <[email protected]>--- fs/xfs/libxfs/xfs_fs.h | 73 ++++++++++++ fs/xfs/libxfs/xfs_ondisk.h | 5 + fs/xfs/libxfs/xfs_parent.c | 35 ++++++ fs/xfs/libxfs/xfs_parent.h | 5 + fs/xfs/xfs_handle.c | 259 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_handle.h | 5 + fs/xfs/xfs_ioctl.c | 6 + fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 92 ++++++++++++++++ 9 files changed, 480 insertions(+), 1 deletion(-)fs/xfs/libxfs/xfs_fs.h +73
-0@@ -840,6 +840,77 @@ struct xfs_commit_range { XFS_EXCHANGE_RANGE_DRY_RUN | \ XFS_EXCHANGE_RANGE_FILE1_WRITTEN) +/* Iterating parent pointers of files. */ + +/* target was the root directory */ +#define XFS_GETPARENTS_OFLAG_ROOT (1U << 0) + +/* Cursor is done iterating pptrs */ +#define XFS_GETPARENTS_OFLAG_DONE (1U << 1) + +#define XFS_GETPARENTS_OFLAGS_ALL (XFS_GETPARENTS_OFLAG_ROOT | \ + XFS_GETPARENTS_OFLAG_DONE) + +#define XFS_GETPARENTS_IFLAGS_ALL (0) + +struct xfs_getparents_rec { +struct xfs_handle gpr_parent;/* Handle to parent */ +__u16 gpr_reclen;/* Length of entire record */ +char gpr_name[];/* Null-terminated filename */ +} __packed; + +/* Iterate through this file's directory parent pointers */ +struct xfs_getparents { +/* +* Structure to track progress in iterating the parent pointers. +* Must be initialized to zeroes before the first ioctl call, and +* not touched by callers after that. +*/ +struct xfs_attrlist_cursor gp_cursor; + +/* Input flags: XFS_GETPARENTS_IFLAG* */ +__u16 gp_iflags; + +/* Output flags: XFS_GETPARENTS_OFLAG* */ +__u16 gp_oflags; + +/* Size of the gp_buffer in bytes */ +__u32 gp_bufsize; + +/* Must be set to zero */ +__u64 __pad; + +/* Pointer to a buffer in which to place xfs_getparents_rec */ +__u64 gp_buffer; +}; + +static inline struct xfs_getparents_rec * +xfs_getparents_first_rec (struct xfs_getparents *gp) +{ +return (struct xfs_getparents_rec *)(uintptr_t )gp->gp_buffer; +} + +static inline struct xfs_getparents_rec * +xfs_getparents_next_rec (struct xfs_getparents *gp, +struct xfs_getparents_rec *gpr) +{ +char *next = ((char *)gpr + gpr->gpr_reclen); +char *end = (char *)(uintptr_t )(gp->gp_buffer + gp->gp_bufsize); + +if (next >= end) +return NULL ; + +return (struct xfs_getparents_rec *)next; +} + +/* Iterate through this file handle's directory parent pointers. */ +struct xfs_getparents_by_handle { +/* Handle to file whose parents we want. */ +struct xfs_handle gph_handle; + +struct xfs_getparents gph_request; +}; +/* * ioctl commands that are used by Linux filesystems */ @@ -875,6 +946,8 @@ struct xfs_commit_range {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +#define XFS_IOC_GETPARENTS _IOWR('X', 62, struct xfs_getparents) +#define XFS_IOC_GETPARENTS_BY_HANDLE _IOWR('X', 63, struct xfs_getparents_by_handle)/* * ioctl commands that replace IRIX syssgi()'s fs/xfs/libxfs/xfs_ondisk.h +5
-0@@ -156,6 +156,11 @@ xfs_check_ondisk_structs(void) XFS_CHECK_OFFSET(struct xfs_efi_log_format_32, efi_extents, 16); XFS_CHECK_OFFSET(struct xfs_efi_log_format_64, efi_extents, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents_rec, 26); + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents, 40); + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents_by_handle, 64); +/* * The v5 superblock format extended several v4 header structures with * additional data. While new fields are only accessible on v5 fs/xfs/libxfs/xfs_parent.c +35
-0@@ -259,3 +259,38 @@ xfs_parent_replacename(xfs_attr_defer_parent (&ppargs->args,XFS_ATTR_DEFER_REPLACE );return 0 ; } + +/* +* Extract parent pointer information from any xattr into @parent_ino/gen. +* The last two parameters can be NULL pointers. +* +* Returns 1 if this is a valid parent pointer; 0 if this is not a parent +* pointer xattr at all; or -EFSCORRUPTED for garbage. +*/ +int +xfs_parent_from_xattr ( +struct xfs_mount *mp, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +xfs_ino_t *parent_ino, +uint32_t *parent_gen) +{ +const struct xfs_parent_rec *rec = value; + +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ; + +if (!xfs_parent_namecheck (attr_flags, name, namelen)) +return -EFSCORRUPTED ; +if (!xfs_parent_valuecheck (mp, value, valuelen)) +return -EFSCORRUPTED ; + +if (parent_ino) + *parent_ino =be64_to_cpu (rec->p_ino); +if (parent_gen) + *parent_gen =be32_to_cpu (rec->p_gen); +return 1 ; +}fs/xfs/libxfs/xfs_parent.h +5
-0@@ -91,4 +91,9 @@ int xfs_parent_replacename(struct xfs_trans *tp, struct xfs_inode *new_dp, const struct xfs_name *new_name, struct xfs_inode *child); +int xfs_parent_from_xattr (struct xfs_mount *mp,unsigned int attr_flags, +const unsigned char *name,unsigned int namelen, +const void *value,unsigned int valuelen, +xfs_ino_t *parent_ino,uint32_t *parent_gen); +#endif /* __XFS_PARENT_H__ */ fs/xfs/xfs_handle.c +235
-0@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* * Copyright (c) 2000-2005 Silicon Graphics, Inc. +* Copyright (c) 2022-2024 Oracle. * All rights reserved. */ #include "xfs.h" @@ -645,3 +646,261 @@ xfs_attrmulti_by_handle(dput (dentry);return error; } + +struct xfs_getparents_ctx { +struct xfs_attr_list_context context; +struct xfs_getparents_by_handle gph; + +/* File to target */ +struct xfs_inode *ip; + +/* Internal buffer where we format records */ +void *krecords; + +/* Last record filled out */ +struct xfs_getparents_rec *lastrec; + +unsigned int count; +}; + +static inline unsigned int +xfs_getparents_rec_sizeof ( +unsigned int namelen) +{ +return round_up (sizeof (struct xfs_getparents_rec ) + namelen +1 , +sizeof (uint32_t )); +} + +static void +xfs_getparents_put_listent ( +struct xfs_attr_list_context *context, +int flags, +unsigned char *name, +int namelen, +void *value, +int valuelen) +{ +struct xfs_getparents_ctx *gpx = + container_of(context, struct xfs_getparents_ctx, context); +struct xfs_inode *ip = context->dp; +struct xfs_mount *mp = ip->i_mount; +struct xfs_getparents *gp = &gpx->gph.gph_request; +struct xfs_getparents_rec *gpr = gpx->krecords + context->firstu; +unsigned short reclen =xfs_getparents_rec_sizeof (namelen); +xfs_ino_t ino; +uint32_t gen; +int ret; + + ret =xfs_parent_from_xattr (mp, flags, name, namelen, value, valuelen, + &ino, &gen); +if (ret <0 ) { +xfs_inode_mark_sick (ip,XFS_SICK_INO_PARENT ); + context->seen_enough = -EFSCORRUPTED ; +return ; + } +if (ret !=1 ) +return ; + +/* +* We found a parent pointer, but we've filled up the buffer. Signal +* to the caller that we did /not/ reach the end of the parent pointer +* recordset. +*/ +if (context->firstu > context->bufsize - reclen) { + context->seen_enough =1 ; +return ; + } + +/* Format the parent pointer directly into the caller buffer. */ + gpr->gpr_reclen = reclen; +xfs_filehandle_init (mp, ino, gen, &gpr->gpr_parent); +memcpy (gpr->gpr_name, name, namelen); + gpr->gpr_name[namelen] =0 ; + +trace_xfs_getparents_put_listent (ip, gp, context, gpr); + + context->firstu += reclen; + gpx->count++; + gpx->lastrec = gpr; +} + +/* Expand the last record to fill the rest of the caller's buffer. */ +static inline void +xfs_getparents_expand_lastrec ( +struct xfs_getparents_ctx *gpx) +{ +struct xfs_getparents *gp = &gpx->gph.gph_request; +struct xfs_getparents_rec *gpr = gpx->lastrec; + +if (!gpx->lastrec) + gpr = gpx->krecords; + + gpr->gpr_reclen = gp->gp_bufsize - ((void *)gpr - gpx->krecords); + +trace_xfs_getparents_expand_lastrec (gpx->ip, gp, &gpx->context, gpr); +} + +static inline void __user *u64_to_uptr (u64 val) +{ +return (void __user *)(uintptr_t )val; +} + +/* Retrieve the parent pointers for a given inode. */ +STATIC int +xfs_getparents ( +struct xfs_getparents_ctx *gpx) +{ +struct xfs_getparents *gp = &gpx->gph.gph_request; +struct xfs_inode *ip = gpx->ip; +struct xfs_mount *mp = ip->i_mount; +size_t bufsize; +int error; + +/* Check size of buffer requested by user */ +if (gp->gp_bufsize >XFS_XATTR_LIST_MAX ) +return -ENOMEM ; +if (gp->gp_bufsize <xfs_getparents_rec_sizeof (1 )) +return -EINVAL ; + +if (gp->gp_iflags & ~XFS_GETPARENTS_IFLAGS_ALL ) +return -EINVAL ; +if (gp->__pad) +return -EINVAL ; + + bufsize =round_down (gp->gp_bufsize,sizeof (uint32_t )); + gpx->krecords =kvzalloc (bufsize,GFP_KERNEL ); +if (!gpx->krecords) { + bufsize =min (bufsize,PAGE_SIZE ); + gpx->krecords =kvzalloc (bufsize,GFP_KERNEL ); +if (!gpx->krecords) +return -ENOMEM ; + } + + gpx->context.dp = ip; + gpx->context.resynch =1 ; + gpx->context.put_listent = xfs_getparents_put_listent; + gpx->context.bufsize = bufsize; +/* firstu is used to track the bytes filled in the buffer */ + gpx->context.firstu =0 ; + +/* Copy the cursor provided by caller */ +memcpy (&gpx->context.cursor, &gp->gp_cursor, +sizeof (struct xfs_attrlist_cursor )); + gpx->count =0 ; + gp->gp_oflags =0 ; + +trace_xfs_getparents_begin (ip, gp, &gpx->context.cursor); + + error =xfs_attr_list (&gpx->context); +if (error) + goto out_free_buf; +if (gpx->context.seen_enough <0 ) { + error = gpx->context.seen_enough; + goto out_free_buf; + } +xfs_getparents_expand_lastrec (gpx); + +/* Update the caller with the current cursor position */ +memcpy (&gp->gp_cursor, &gpx->context.cursor, +sizeof (struct xfs_attrlist_cursor )); + +/* Is this the root directory? */ +if (ip->i_ino == mp->m_sb.sb_rootino) + gp->gp_oflags |=XFS_GETPARENTS_OFLAG_ROOT ; + +if (gpx->context.seen_enough ==0 ) { +/* +* If we did not run out of buffer space, then we reached the +* end of the pptr recordset, so set the DONE flag. +*/ + gp->gp_oflags |=XFS_GETPARENTS_OFLAG_DONE ; + }else if (gpx->count ==0 ) { +/* +* If we ran out of buffer space before copying any parent +* pointers at all, the caller's buffer was too short. Tell +* userspace that, erm, the message is too long. +*/ + error = -EMSGSIZE ; + goto out_free_buf; + } + +trace_xfs_getparents_end (ip, gp, &gpx->context.cursor); + +ASSERT (gpx->context.firstu <= gpx->gph.gph_request.gp_bufsize); + +/* Copy the records to userspace. */ +if (copy_to_user (u64_to_uptr (gpx->gph.gph_request.gp_buffer), + gpx->krecords, gpx->context.firstu)) + error = -EFAULT ; + +out_free_buf: +kvfree (gpx->krecords); + gpx->krecords =NULL ; +return error; +} + +/* Retrieve the parents of this file and pass them back to userspace. */ +int +xfs_ioc_getparents ( +struct file *file, +struct xfs_getparents __user *ureq) +{ +struct xfs_getparents_ctx gpx = { + .ip =XFS_I (file_inode (file)), + }; +struct xfs_getparents *kreq = &gpx.gph.gph_request; +struct xfs_mount *mp = gpx.ip->i_mount; +int error; + +if (!capable (CAP_SYS_ADMIN )) +return -EPERM ; +if (!xfs_has_parent (mp)) +return -EOPNOTSUPP ; +if (copy_from_user (kreq, ureq,sizeof (*kreq))) +return -EFAULT ; + + error =xfs_getparents (&gpx); +if (error) +return error; + +if (copy_to_user (ureq, kreq,sizeof (*kreq))) +return -EFAULT ; + +return 0 ; +} + +/* Retrieve the parents of this file handle and pass them back to userspace. */ +int +xfs_ioc_getparents_by_handle ( +struct file *file, +struct xfs_getparents_by_handle __user *ureq) +{ +struct xfs_getparents_ctx gpx = { }; +struct xfs_inode *ip =XFS_I (file_inode (file)); +struct xfs_mount *mp = ip->i_mount; +struct xfs_getparents_by_handle *kreq = &gpx.gph; +struct dentry *dentry;
fs/xfs/xfs_handle.c xfs_attrmulti_by_handle( · line 892
Quoted 10 lines
fs/xfs/xfs_handle.c +10
-0+int error; + +if (!capable (CAP_SYS_ADMIN )) +return -EPERM ; +if (!xfs_has_parent (mp)) +return -EOPNOTSUPP ; +if (copy_from_user (kreq, ureq,sizeof (*kreq))) +return -EFAULT ; + + dentry =xfs_khandle_to_dentry (file, &kreq->gph_handle);
I noticed a couple of things while doing more testing here -- first, xfs_khandle_to_dentry doesn't check that the handle fsid actually matches this filesystem, and AFAICT *nothing* actually checks that. So I guess that's a longstanding weakness of handle validation, and we probably haven't gotten any reports because what's the chance that you'll get lucky with an ino/gen from a different filesystem?
(Pretty good what with golden images proliferating, I'd say...)
The second thing is that exportfs_decode_fh does too much work here -- if the handle references a directory, it'll walk up the directory tree to the root to try to reconnect the dentry paths. For GETPARENTS we don't care about that since we're not doing anything with dentries. Walking upwards in the directory tree is extra work that doesn't change the results.
Worse yet, if there's a loop in in the directory tree due to dotdot damage or whatnot, this can livelock the system. This is unfortunate for xfs_scrub because it'll use GETPARENTS to try to report the path of a file that it wants to repair ... but it might not have checked those parents. So I really don't want GETPARENTS to be creating a bunch of dentries and reconnecting paths and whatnot.
What we really want, I think, is some basic handle validation and then a call to something like xfs_nfs_get_inode.
if (!S_ISDIR(VFS_I(ip)->i_mode)) return -ENOTDIR; if (memcmp(&handle->ha_fsid, mp->m_fixedfsid, sizeof(struct xfs_fsid))) return -ESTALE; if (handle->ha_fid.fid_len != xfs_filehandle_fid_len()) return -EINVAL; inode = xfs_nfs_get_inode(mp->m_super, handle->ha_fid.fid_ino, handle->ha_fid.fid_gen); if (IS_ERR(inode)) return PTR_ERR(inode); gpx.ip = XFS_I(inode); error = xfs_getparents(&gpx);
And yes, I'll add a big comment explaining why we don't use the regular handle functions here.
--D
Quoted 183 lines
fs/xfs/xfs_handle.c +14
-0+if (IS_ERR (dentry)) +return PTR_ERR (dentry); + + gpx.ip =XFS_I (dentry->d_inode); + error =xfs_getparents (&gpx); +dput (dentry); +if (error) +return error; + +if (copy_to_user (ureq, kreq,sizeof (*kreq))) +return -EFAULT ; + +return 0 ; +}fs/xfs/xfs_handle.h +5
-0@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* * Copyright (c) 2000-2005 Silicon Graphics, Inc. +* Copyright (c) 2022-2024 Oracle. * All rights reserved. */ #ifndef __XFS_HANDLE_H__@@ -25,4 +26,8 @@ int xfs_ioc_attr_list(struct xfs_inode *dp, void __user *ubuf,struct dentry *xfs_handle_to_dentry (struct file *parfilp,void __user *uhandle,u32 hlen); +int xfs_ioc_getparents (struct file *file,struct xfs_getparents __user *arg); +int xfs_ioc_getparents_by_handle (struct file *file, +struct xfs_getparents_by_handle __user *arg); +#endif /* __XFS_HANDLE_H__ */ fs/xfs/xfs_ioctl.c +5
-1@@ -35,6 +35,7 @@#include "xfs_health.h" #include "xfs_reflink.h" #include "xfs_ioctl.h" +#include "xfs_xattr.h" #include "xfs_rtbitmap.h" #include "xfs_file.h" #include "xfs_exchrange.h" @@ -1542,7 +1543,10 @@ xfs_file_ioctl(case XFS_IOC_FSGETXATTRA :return xfs_ioc_fsgetxattra (ip, arg); - +case XFS_IOC_GETPARENTS : +return xfs_ioc_getparents (filp, arg); +case XFS_IOC_GETPARENTS_BY_HANDLE : +return xfs_ioc_getparents_by_handle (filp, arg);case XFS_IOC_GETBMAP :case XFS_IOC_GETBMAPA :case XFS_IOC_GETBMAPX :fs/xfs/xfs_trace.c +1
-0@@ -41,6 +41,7 @@#include "xfs_bmap.h" #include "xfs_exchmaps.h" #include "xfs_exchrange.h" +#include "xfs_parent.h" /* * We include this last to have the helpers above available for the trace fs/xfs/xfs_trace.h +92
-0@@ -87,6 +87,9 @@ struct xfs_bmap_intent;struct xfs_exchmaps_intent ;struct xfs_exchmaps_req ;struct xfs_exchrange ; +struct xfs_getparents ; +struct xfs_parent_irec ; +struct xfs_attrlist_cursor_kern ;#define XFS_ATTR_FILTER_FLAGS \ { XFS_ATTR_ROOT, "ROOT" }, \@@ -5158,6 +5161,95 @@ TRACE_EVENT(xfs_exchmaps_delta_nextents, __entry->d_nexts1, __entry->d_nexts2) ); +DECLARE_EVENT_CLASS(xfs_getparents_rec_class, + TP_PROTO(struct xfs_inode *ip, const struct xfs_getparents *ppi, + const struct xfs_attr_list_context *context, + const struct xfs_getparents_rec *pptr), + TP_ARGS(ip, ppi, context, pptr), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned int, firstu) + __field(unsigned short, reclen) + __field(unsigned int, bufsize) + __field(xfs_ino_t, parent_ino) + __field(unsigned int, parent_gen) + __string(name, pptr->gpr_name) + ), + TP_fast_assign( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->firstu = context->firstu; + __entry->reclen = pptr->gpr_reclen; + __entry->bufsize = ppi->gp_bufsize; + __entry->parent_ino = pptr->gpr_parent.ha_fid.fid_ino; + __entry->parent_gen = pptr->gpr_parent.ha_fid.fid_gen; + __assign_str(name, pptr->gpr_name); + ), + TP_printk("dev %d:%d ino 0x%llx firstu %u reclen %u bufsize %u parent_ino 0x%llx parent_gen 0x%x name '%s'", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->ino, + __entry->firstu, + __entry->reclen, + __entry->bufsize, + __entry->parent_ino, + __entry->parent_gen, + __get_str(name)) +) +#define DEFINE_XFS_GETPARENTS_REC_EVENT (name) \ +DEFINE_EVENT(xfs_getparents_rec_class, name, \ + TP_PROTO(struct xfs_inode *ip, const struct xfs_getparents *ppi, \ + const struct xfs_attr_list_context *context, \ + const struct xfs_getparents_rec *pptr), \ + TP_ARGS(ip, ppi, context, pptr)) +DEFINE_XFS_GETPARENTS_REC_EVENT (xfs_getparents_put_listent); +DEFINE_XFS_GETPARENTS_REC_EVENT (xfs_getparents_expand_lastrec); + +DECLARE_EVENT_CLASS (xfs_getparents_class, + TP_PROTO(struct xfs_inode *ip, const struct xfs_getparents *ppi, + const struct xfs_attrlist_cursor_kern *cur), +TP_ARGS (ip, ppi, cur), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned short, iflags) + __field(unsigned short, oflags) + __field(unsigned int, bufsize) + __field(unsigned int, hashval) + __field(unsigned int, blkno) + __field(unsigned int, offset) +__field (int, initted) + ), +TP_fast_assign ( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->iflags = ppi->gp_iflags; + __entry->oflags = ppi->gp_oflags; + __entry->bufsize = ppi->gp_bufsize; + __entry->hashval = cur->hashval; + __entry->blkno = cur->blkno; + __entry->offset = cur->offset; + __entry->initted = cur->initted; + ), +TP_printk ("dev %d:%d ino 0x%llx iflags 0x%x oflags 0x%x bufsize %u cur_init? %d hashval 0x%x blkno %u offset %u" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->iflags, + __entry->oflags, + __entry->bufsize, + __entry->initted, + __entry->hashval, + __entry->blkno, + __entry->offset) +) +#define DEFINE_XFS_GETPARENTS_EVENT (name) \ +DEFINE_EVENT(xfs_getparents_class, name, \ + TP_PROTO(struct xfs_inode *ip, const struct xfs_getparents *ppi, \ + const struct xfs_attrlist_cursor_kern *cur), \ + TP_ARGS(ip, ppi, cur)) +DEFINE_XFS_GETPARENTS_EVENT (xfs_getparents_begin); +DEFINE_XFS_GETPARENTS_EVENT (xfs_getparents_end); +#endif /* _TRACE_XFS_H */ #undef TRACE_INCLUDE_PATH
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
[full quote deleted. It took me about a minute of scrolling to find the actual contents, *sigh*]
Quoted 3 lines · Darrick J. Wong
I noticed a couple of things while doing more testing here -- first,
xfs_khandle_to_dentrydoesn't check that the handle fsid actually matches this filesystem, and AFAICT *nothing* actually checks that.
Yes. Userspace better have resolved that, as the ioctl only works on the given file system, so libhandle has to resolve it before even calling the ioctl.
Quoted 3 lines
So I guess that's a longstanding weakness of handle validation, and we probably haven't gotten any reports because what's the chance that you'll get lucky with an ino/gen from a different filesystem?
Not really, see above.
Quoted 6 lines
The second thing is that
exportfs_decode_fhdoes too much work here -- if the handle references a directory, it'll walk up the directory tree to the root to try to reconnect the dentry paths. For GETPARENTS we don't care about that since we're not doing anything with dentries. Walking upwards in the directory tree is extra work that doesn't change the results.
In theory no one cares as all operations work just fine with disconnected dentries, and exportfs_decode_fh doesn't do these checks unless the accpetable parameter is passed to it. The real question is why we (which in this case means 15 years younger me) decided back then we want this checking for XFS handle operations? I can't really think of one right now..
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 11 lines · Christoph Hellwig
[full quote deleted. It took me about a minute of scrolling to find the actual contents, *sigh*]
Quoted 3 lines · Darrick J. Wong
I noticed a couple of things while doing more testing here -- first,
xfs_khandle_to_dentrydoesn't check that the handle fsid actually matches this filesystem, and AFAICT *nothing* actually checks that.Yes. Userspace better have resolved that, as the ioctl only works on the given file system, so libhandle has to resolve it before even calling the ioctl.
True, libhandle is a very nice wrapper for the kernel ioctls. I wish Linux projects did that more often. But suppose you're calling the ioctls directly without libhandle and mess it up?
Quoted 13 lines
Quoted 3 lines
So I guess that's a longstanding weakness of handle validation, and we probably haven't gotten any reports because what's the chance that you'll get lucky with an ino/gen from a different filesystem?
Not really, see above.
Quoted 6 lines
The second thing is that
exportfs_decode_fhdoes too much work here -- if the handle references a directory, it'll walk up the directory tree to the root to try to reconnect the dentry paths. For GETPARENTS we don't care about that since we're not doing anything with dentries. Walking upwards in the directory tree is extra work that doesn't change the results.
Quoted 6 lines
In theory no one cares as all operations work just fine with disconnected dentries, and
exportfs_decode_fhdoesn't do these checks unless the accpetable parameter is passed to it. The real question is why we (which in this case means 15 years younger me) decided back then we want this checking for XFS handle operations? I can't really think of one right now..
Me neither. Though at this point there are a lot of filesystems that implement ->get_parent, so I think removing XFS's will need a discussion at least on linux-xfs, if not fsdevel. In the meantime, getparents can do minimal validation + iget for now and if it makes sense to port it back to xfs_khandle_to_dentry, I can do that easily.
(FWIW turning off reconnection would likely fix some of the annoying behaviors of xfs_scrub where it tries to open a dir to scan it and then sprays dmesg with errors from unrelated parents as it stumbles over reconnection only to fail the open, at which point it falls back to scrubbing by handle anyway.)
--D
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 3 lines · Darrick J. Wong
True, libhandle is a very nice wrapper for the kernel ioctls. I wish Linux projects did that more often. But suppose you're calling the ioctls directly without libhandle and mess it up?
The you get different inodes back. Not really any different from pointing your path name based code to the wrong fs or directory, is it?
Quoted 7 lines
Quoted 6 lines
In theory no one cares as all operations work just fine with disconnected dentries, and
exportfs_decode_fhdoesn't do these checks unless the accpetable parameter is passed to it. The real question is why we (which in this case means 15 years younger me) decided back then we want this checking for XFS handle operations? I can't really think of one right now..
Quoted 5 lines
Me neither. Though at this point there are a lot of filesystems that implement ->get_parent, so I think removing XFS's will need a discussion at least on linux-xfs, if not fsdevel. In the meantime, getparents can do minimal validation + iget for now and if it makes sense to port it back to
xfs_khandle_to_dentry, I can do that easily.
Uhh, I'm not advocating for removing ->get_parent at all. We actually do need that for security on NFS, where the file handles are used undernath pathname based operations.
And it turns out my previous analysis wasn't quite sport on. The exportfs code always reconnects directories, because we basically have to, not connecting them would make the VFS locking scheme not work.
But as we never generate the file handles that encode the parent we already never connect files to their parent directory anyway.
OTOH we should be able to optimize ->get_parent a bit with parent pointers, as we can find the name in the parent directory for a directory instead of doing linear scans in the parent directory. (for non-directory files we currenty don't fully connect anwyay)
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
True, libhandle is a very nice wrapper for the kernel ioctls. I wish Linux projects did that more often. But suppose you're calling the ioctls directly without libhandle and mess it up?
The you get different inodes back. Not really any different from pointing your path name based code to the wrong fs or directory, is it?
I suppose not. But why bother setting the fsid at all, then?
Quoted 13 lines
Quoted 7 lines
Quoted 6 lines
In theory no one cares as all operations work just fine with disconnected dentries, and
exportfs_decode_fhdoesn't do these checks unless the accpetable parameter is passed to it. The real question is why we (which in this case means 15 years younger me) decided back then we want this checking for XFS handle operations? I can't really think of one right now..Quoted 5 lines
Me neither. Though at this point there are a lot of filesystems that implement ->get_parent, so I think removing XFS's will need a discussion at least on linux-xfs, if not fsdevel. In the meantime, getparents can do minimal validation + iget for now and if it makes sense to port it back to
xfs_khandle_to_dentry, I can do that easily.
Quoted 3 lines
Uhh, I'm not advocating for removing ->get_parent at all. We actually do need that for security on NFS, where the file handles are used undernath pathname based operations.
Ahh, I wasn't aware of that, beyond a sense that "a lot of NFS-exportable fses do this, so there's likely a general desire for this to be wired up."
Quoted 4 lines
And it turns out my previous analysis wasn't quite sport on. The exportfs code always reconnects directories, because we basically have to, not connecting them would make the VFS locking scheme not work.
Noted.
Quoted 2 lines
But as we never generate the file handles that encode the parent we already never connect files to their parent directory anyway.
I pondered whether or not we should encode parent info in a regular file's handle. Would that result in an invalid handle if the file gets moved to another directory? That doesn't seem to fit with the behavior that fds remain attached to the file even if it gets moved/deleted.
Quoted 4 lines
OTOH we should be able to optimize ->get_parent a bit with parent pointers, as we can find the name in the parent directory for a directory instead of doing linear scans in the parent directory. (for non-directory files we currenty don't fully connect anwyay)
<nod> But does exportfs actually want parent info for a nondirectory? There aren't any stubs or XXX/FIXME comments, and I've never heard any calls (at least on fsdevel) for that functionality.
--D
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 11 lines · Darrick J. Wong
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
True, libhandle is a very nice wrapper for the kernel ioctls. I wish Linux projects did that more often. But suppose you're calling the ioctls directly without libhandle and mess it up?
The you get different inodes back. Not really any different from pointing your path name based code to the wrong fs or directory, is it?
I suppose not. But why bother setting the fsid at all, then?
I suspect that's a leftover from IRIX where the by handle operations weren't ioctls tied to a specific file system.
Quoted 5 lines
Quoted 2 lines
But as we never generate the file handles that encode the parent we already never connect files to their parent directory anyway.
I pondered whether or not we should encode parent info in a regular file's handle.
We shouldn't. It's a really a NFS thing.
Quoted 2 lines
Would that result in an invalid handle if the file gets moved to another directory?
Yes.
Quoted 2 lines
That doesn't seem to fit with the behavior
that fds remain attached to the file even if it gets moved/deleted.
Exactly.
Quoted 9 lines
Quoted 5 lines
OTOH we should be able to optimize ->get_parent a bit with parent pointers, as we can find the name in the parent directory for a directory instead of doing linear scans in the parent directory. (for non-directory files we currenty don't fully connect anwyay)
<nod> But does exportfs actually want parent info for a nondirectory? There aren't any stubs or XXX/FIXME comments, and I've never heard any calls (at least on fsdevel) for that functionality.
It doesn't. It would avoid having disconnected dentries, but disconnected non-directory dentries aren't really a problem.
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
True, libhandle is a very nice wrapper for the kernel ioctls. I wish Linux projects did that more often. But suppose you're calling the ioctls directly without libhandle and mess it up?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
The you get different inodes back. Not really any different from pointing your path name based code to the wrong fs or directory, is it?
I suppose not. But why bother setting the fsid at all, then?
I suspect that's a leftover from IRIX where the by handle operations weren't ioctls tied to a specific file system.
Oh, so on Irix a program could call the kernel with *only* the handle and no fd? I wasn't aware of that, but most of my exposure to Irix was wowwwing over the 3D File Explorer in _Jurassic Park_ and later an old Indigo that someone donated to the high school. ;)
Ok, I'll drop the fsid checking code entirely.
Quoted 12 lines
Quoted 5 lines
Quoted 2 lines
But as we never generate the file handles that encode the parent we already never connect files to their parent directory anyway.
I pondered whether or not we should encode parent info in a regular file's handle.
We shouldn't. It's a really a NFS thing.
Quoted 2 lines
Would that result in an invalid handle if the file gets moved to another directory?
Quoted 6 lines
Yes.
Quoted 2 lines
That doesn't seem to fit with the behavior
that fds remain attached to the file even if it gets moved/deleted.Exactly.
<nod>
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
OTOH we should be able to optimize ->get_parent a bit with parent pointers, as we can find the name in the parent directory for a directory instead of doing linear scans in the parent directory. (for non-directory files we currenty don't fully connect anwyay)
Quoted 6 lines
Quoted 3 lines
<nod> But does exportfs actually want parent info for a nondirectory? There aren't any stubs or XXX/FIXME comments, and I've never heard any calls (at least on fsdevel) for that functionality.
It doesn't. It would avoid having disconnected dentries, but disconnected non-directory dentries aren't really a problem.
For directories, I think the dotdot lookup is much cheaper than scanning the attrs to find the first nongarbage XFS_ATTR_PARENT entry.
--D
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 9 lines · Darrick J. Wong
Quoted 6 lines
Quoted 3 lines
<nod> But does exportfs actually want parent info for a nondirectory? There aren't any stubs or XXX/FIXME comments, and I've never heard any calls (at least on fsdevel) for that functionality.
It doesn't. It would avoid having disconnected dentries, but disconnected non-directory dentries aren't really a problem.
For directories, I think the dotdot lookup is much cheaper than scanning the attrs to find the first nongarbage
XFS_ATTR_PARENTentry.
It is.
But I was confused again, it's been a while since I worked on that code..
We do the full reconnection for non-directories if NFSD asks for it (the XFS or VFS handle code won't hit this because our acceptable callback
always returns true). That code does a readdir on the parent and
returns the name when it finds the inode number. For files without crazy number of hardlinks just looking over the parent pointers would be a lot more efficient for that.
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 13 lines · Christoph Hellwig
Quoted 9 lines · Darrick J. Wong
Quoted 6 lines
Quoted 3 lines
<nod> But does exportfs actually want parent info for a nondirectory? There aren't any stubs or XXX/FIXME comments, and I've never heard any calls (at least on fsdevel) for that functionality.
It doesn't. It would avoid having disconnected dentries, but disconnected non-directory dentries aren't really a problem.
For directories, I think the dotdot lookup is much cheaper than scanning the attrs to find the first nongarbage
XFS_ATTR_PARENTentry.It is.
Quoted 8 lines
But I was confused again, it's been a while since I worked on that code..
We do the full reconnection for non-directories if NFSD asks for it (the XFS or VFS handle code won't hit this because our acceptable callback
always returns true). That code does a readdir on the parent and
returns the name when it finds the inode number. For files without crazy number of hardlinks just looking over the parent pointers would be a lot more efficient for that.
Ohhhh, does that happens outside of XFS then? No wonder I couldn't find what you were talking about. Ok I'll go look some more.
--D
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 2 lines · Darrick J. Wong
Ohhhh, does that happens outside of XFS then? No wonder I couldn't find what you were talking about. Ok I'll go look some more.
Yes. get_name() in fs/exportfs/expfs.c.
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong
Ohhhh, does that happens outside of XFS then? No wonder I couldn't find what you were talking about. Ok I'll go look some more.
Yes.
get_name()infs/exportfs/expfs.c.
Hmm. Implementing a custom ->get_name for pptrs would work well for child files with low hardlink counts. Certainly there are probably a lot more large directories than files that are hardlinked many many times. At what point does it become cheaper to scan the directory?
--D
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 11 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong
Ohhhh, does that happens outside of XFS then? No wonder I couldn't find what you were talking about. Ok I'll go look some more.
Yes.
get_name()infs/exportfs/expfs.c.Hmm. Implementing a custom ->get_name for pptrs would work well for child files with low hardlink counts. Certainly there are probably a lot more large directories than files that are hardlinked many many times. At what point does it become cheaper to scan the directory?
Note that despite my previous confusion get_name is also called for directories to find the actual name they have in their parent.
An easy conservative choice would be to always look at the parent pointers for nlink==1.
All of that is for later, I don't want to delay the parent pointers series even further.
Re: [PATCH 27/32] xfs: Add parent pointer ioctls
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong
Ohhhh, does that happens outside of XFS then? No wonder I couldn't find what you were talking about. Ok I'll go look some more.
Yes.
get_name()infs/exportfs/expfs.c.Hmm. Implementing a custom ->get_name for pptrs would work well for child files with low hardlink counts. Certainly there are probably a lot more large directories than files that are hardlinked many many times. At what point does it become cheaper to scan the directory?
Quoted 8 lines
Note that despite my previous confusion
get_nameis also called for directories to find the actual name they have in their parent.An easy conservative choice would be to always look at the parent pointers for nlink==1.
All of that is for later, I don't want to delay the parent pointers series even further.
It won't; I'll need to test the default ->get_name implementation so that I can compare it to the new one, which means I'll have to figure out /how/ to test that.
--D
PATCH 28/32 xfs: don't remove the attr fork when parent pointers are enabled
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
When an inode is removed, it may also cause the attribute fork to be removed if it is the last attribute. This transaction gets flushed to the log, but if the system goes down before we could inactivate the symlink, the log recovery tries to inactivate this inode (since it is on the unlinked list) but the verifier trips over the remote value and leaks it.
Hence we ended up with a file in this odd state on a "clean" mount. The "obvious" fix is to prohibit erasure of the attr fork to avoid tripping over the verifiers when pptrs are enabled.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +4 -2
@@ -887,7 +887,8 @@ xfs_attr_sf_removename(*/ if (totsize ==sizeof (struct xfs_attr_sf_hdr ) &&xfs_has_attr2 (mp) && (dp->i_df.if_format !=XFS_DINODE_FMT_BTREE ) && - !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE ))) { + !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE )) && + !xfs_has_parent (mp)) {xfs_attr_fork_remove (dp, args->trans); }else {xfs_idata_realloc (dp, -size,XFS_ATTR_FORK );
@@ -896,7 +897,8 @@ xfs_attr_sf_removename(ASSERT (totsize >sizeof (struct xfs_attr_sf_hdr ) || (args->op_flags &XFS_DA_OP_ADDNAME ) || !xfs_has_attr2 (mp) || - dp->i_df.if_format ==XFS_DINODE_FMT_BTREE ); + dp->i_df.if_format ==XFS_DINODE_FMT_BTREE || +xfs_has_parent (mp));xfs_trans_log_inode (args->trans, dp,XFS_ILOG_CORE |XFS_ILOG_ADATA ); }
Re: [PATCH 28/32] xfs: don't remove the attr fork when parent pointers are enabled
Looks good:
PATCH 29/32 xfs: Add the parent pointer support to the superblock version 5.
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the parent pointer superblock flag so that we can actually mount filesystems with this feature enabled.
--- fs/xfs/libxfs/xfs_format.h | 1 + fs/xfs/libxfs/xfs_fs.h | 2 ++ fs/xfs/libxfs/xfs_sb.c | 4 ++++ fs/xfs/xfs_super.c | 4 ++++ 4 files changed, 11 insertions(+)
fs/xfs/libxfs/xfs_format.h +1 -0
@@ -374,6 +374,7 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)/* needs xfs_repair */ #define XFS_SB_FEAT_INCOMPAT_NREXT64 (1 << 5)/* large extent counters */ #define XFS_SB_FEAT_INCOMPAT_EXCHRANGE (1 << 6)/* exchangerange supported */ +#define XFS_SB_FEAT_INCOMPAT_PARENT (1 << 7)/* parent pointers */ #define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE | \ XFS_SB_FEAT_INCOMPAT_SPINODES | \
fs/xfs/libxfs/xfs_fs.h +2 -0
@@ -241,6 +241,8 @@ typedef struct xfs_fsop_resblks {#define XFS_FSOP_GEOM_FLAGS_NREXT64 (1 << 23)/* large extent counters */ #define XFS_FSOP_GEOM_FLAGS_EXCHANGE_RANGE (1 << 24)/* exchange range */ +#define XFS_FSOP_GEOM_FLAGS_PARENT (1U << 30)/* parent pointers */ +/* * Minimum and maximum sizes need for growth checks. *
fs/xfs/libxfs/xfs_sb.c +4 -0
@@ -178,6 +178,8 @@ xfs_sb_version_to_features( features |=XFS_FEAT_NREXT64 ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_EXCHRANGE ) features |=XFS_FEAT_EXCHANGE_RANGE ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_PARENT ) + features |=XFS_FEAT_PARENT ;return features; }
@@ -1254,6 +1256,8 @@ xfs_fs_geometry( geo->flags |=XFS_FSOP_GEOM_FLAGS_BIGTIME ;if (xfs_has_inobtcounts (mp)) geo->flags |=XFS_FSOP_GEOM_FLAGS_INOBTCNT ; +if (xfs_has_parent (mp)) + geo->flags |=XFS_FSOP_GEOM_FLAGS_PARENT ;if (xfs_has_sector (mp)) { geo->flags |=XFS_FSOP_GEOM_FLAGS_SECTOR ; geo->logsectsize = sbp->sb_logsectsize;
fs/xfs/xfs_super.c +4 -0
@@ -1733,6 +1733,10 @@ xfs_fs_fill_super(xfs_warn (mp,"EXPERIMENTAL exchange-range feature enabled. Use at your own risk!" ); +if (xfs_has_parent (mp)) +xfs_warn (mp, +"EXPERIMENTAL parent pointer feature enabled. Use at your own risk!" ); + error =xfs_mountfs (mp);if (error) goto out_filestream_unmount;
Re: [PATCH 29/32] xfs: Add the parent pointer support to the superblock version 5.
Quoted 4 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
Add the parent pointer superblock flag so that we can actually mount filesystems with this feature enabled.
The subjcet reads a little weird. What about
"add a incompat feature bit for parent pointers" ?
Re: [PATCH 29/32] xfs: Add the parent pointer support to the superblock version 5.
Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
From: Allison Henderson <[email protected]>
Add the parent pointer superblock flag so that we can actually mount filesystems with this feature enabled.
The subjcet reads a little weird. What about
"add a incompat feature bit for parent pointers" ?
Changed to:
"xfs: add a incompat feature bit for parent pointers
"Create an incompat feature bit and a fs geometry flag so that we can enable the feature in the ondisk superblock and advertise its existence to userspace."
Also I'll change the bit to 1<<25; I was using 1U<<30 to avoid collisions with the other patchsets.
--D
PATCH 30/32 xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Dave and I were discussing some recent test regressions as a result of me turning on nrext64=1 on realtime filesystems, when we noticed that the minimum log size of a 32M filesystem jumped from 954 blocks to 4287 blocks.
Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size contains the maximum estimated amount of space needed for a local format xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES, which requires units of blocks. This has resulted in an overestimation of the minimum log size over the years.
We should nominally correct this, but there's a backwards compatibility problem -- if we enable it now, the minimum log size will decrease. If a corrected mkfs formats a filesystem with this new smaller log size, a user will encounter mount failures on an uncorrected kernel due to the larger minimum log size computations there.
Therefore, turn this on for parent pointers because it wasn't merged at all upstream when this issue was discovered.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +32 -0
@@ -16,6 +16,29 @@#include "xfs_bmap_btree.h" #include "xfs_trace.h" +/* +* Shortly after enabling the large extents count feature in 2023, longstanding +* bugs were found in the code that computes the minimum log size. Luckily, +* the bugs resulted in over-estimates of that size, so there's no impact to +* existing users. However, we don't want to reduce the minimum log size +* because that can create the situation where a newer mkfs writes a new +* filesystem that an older kernel won't mount. +* +* Therefore, we only may correct the computation starting with filesystem +* features that didn't exist in 2023. In other words, only turn this on if +* the filesystem has parent pointers. +* +* This function can be called before the XFS_HAS_* flags have been set up, +* (e.g. mkfs) so we must check the ondisk superblock. +*/ +static inline bool +xfs_want_minlogsize_fixes ( +struct xfs_sb *sb) +{ +return xfs_sb_is_v5 (sb) && +xfs_sb_has_incompat_feature (sb,XFS_SB_FEAT_INCOMPAT_PARENT ); +} +/* * Calculate the maximum length in bytes that would be required for a local * attribute value as large attributes out of line are not logged.
@@ -31,6 +54,15 @@ xfs_log_calc_max_attrsetm_res(MAXNAMELEN -1 ; nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); nblks +=XFS_B_TO_FSB (mp, size); + +/* +* If the feature set is new enough, correct a unit conversion error in +* the xattr transaction reservation code that resulted in oversized +* minimum log size computations. +*/ +if (xfs_want_minlogsize_fixes (&mp->m_sb)) + size =XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); return M_RES(mp)->tr_attrsetm.tr_logres +
Re: [PATCH 30/32] xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
Looks good:
PATCH 31/32 xfs: drop compatibility minimum log size computations for reflink
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Let's also drop the oversized minimum log computations for reflink and rmap that were the result of bugs introduced many years ago.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +14 -0
@@ -24,6 +24,11 @@* because that can create the situation where a newer mkfs writes a new * filesystem that an older kernel won't mount. * +* Several years prior, we also discovered that the transaction reservations +* for rmap and reflink operations were unnecessarily large. That was fixed, +* but the minimum log size computation was left alone to avoid the +* compatibility problems noted above. Fix that too. +* * Therefore, we only may correct the computation starting with filesystem * features that didn't exist in 2023. In other words, only turn this on if * the filesystem has parent pointers.
@@ -80,6 +85,15 @@ xfs_log_calc_trans_resv_for_minlogblocks( {unsigned int rmap_maxlevels = mp->m_rmap_maxlevels; +/* +* If the feature set is new enough, drop the oversized minimum log +* size computation introduced by the original reflink code. +*/ +if (xfs_want_minlogsize_fixes (&mp->m_sb)) { +xfs_trans_resv_calc (mp, resv); +return ; + } +/* * In the early days of rmap+reflink, we always set the rmap maxlevels * to 9 even if the AG was small enough that it would never grow to
Re: [PATCH 31/32] xfs: drop compatibility minimum log size computations for reflink
Looks good:
PATCH 32/32 xfs: enable parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add parent pointers to the list of supported features.
--- fs/xfs/libxfs/xfs_format.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +2 -1
@@ -382,7 +382,8 @@ xfs_sb_has_ro_compat_feature( XFS_SB_FEAT_INCOMPAT_BIGTIME | \ XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR | \ XFS_SB_FEAT_INCOMPAT_NREXT64 | \ - XFS_SB_FEAT_INCOMPAT_EXCHRANGE) + XFS_SB_FEAT_INCOMPAT_EXCHRANGE | \ + XFS_SB_FEAT_INCOMPAT_PARENT)#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
Re: [PATCH 32/32] xfs: enable parent pointers
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add parent pointers to the list of supported features.
Any reason to split this from actually adding the feature bit?
Otherwise looks good:
Re: [PATCH 32/32] xfs: enable parent pointers
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add parent pointers to the list of supported features.
Any reason to split this from actually adding the feature bit?
Bisection. If we don't add the bit to XFS_SB_FEAT_INCOMPAT_ALL, then the kernel won't mount the filesystem.
Nowadays I think we could define the xfs_has_foo helpers at the start and only add the superblock feature bit and the code that sets XFS_FEAT_FOO in the final patch.
BUuut.... this patchset predates the xfs_has_foo helpers and I haven't fully adjusted to that yet.
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
PATCH 6/9 xfs: scrubbing for parent pointers
In reply to Darrick J. WongHi all,
Teach online fsck to use parent pointers to assist in checking directories, parent pointers, extended attributes, and link counts.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-pptrs
---
Commits in this patchset:
* xfs: check dirents have parent pointers
* xfs: deferred scrub of dirents
* xfs: scrub parent pointers
* xfs: deferred scrub of parent pointers
* xfs: walk directory parent pointers to determine backref count
* xfs: check parent pointer xattrs when scrubbing
* xfs: salvage parent pointers when rebuilding xattr structures
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_parent.c | 22 + fs/xfs/libxfs/xfs_parent.h | 5 fs/xfs/scrub/attr.c | 8 fs/xfs/scrub/attr_repair.c | 34 ++ fs/xfs/scrub/common.h | 1 fs/xfs/scrub/dir.c | 342 +++++++++++++++++++++ fs/xfs/scrub/nlinks.c | 82 +++++ fs/xfs/scrub/nlinks_repair.c | 2 fs/xfs/scrub/parent.c | 678 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/readdir.c | 78 +++++ fs/xfs/scrub/readdir.h | 3 fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 103 ++++++ 14 files changed, 1348 insertions(+), 13 deletions(-)
PATCH 1/7 xfs: check dirents have parent pointers
From: Darrick J. Wong <[email protected]>
If the fs has parent pointers, we need to check that each child dirent points to a file that has a parent pointer pointing back at us.
--- fs/xfs/libxfs/xfs_parent.c | 22 +++++++++ fs/xfs/libxfs/xfs_parent.h | 5 ++ fs/xfs/scrub/dir.c | 112 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 138 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_parent.c +22 -0
@@ -294,3 +294,25 @@ xfs_parent_from_xattr( *parent_gen =be32_to_cpu (rec->p_gen);return 1 ; } + +/* +* Look up a parent pointer record (@parent_name -> @pptr) of @ip. +* +* Caller must hold at least ILOCK_SHARED. The scratchpad need not be +* initialized. +* +* Returns 0 if the pointer is found, -ENOATTR if there is no match, or a +* negative errno. +*/ +int +xfs_parent_lookup ( +struct xfs_trans *tp, +struct xfs_inode *ip, +const struct xfs_name *parent_name, +struct xfs_parent_rec *pptr, +struct xfs_da_args *scratch) +{ +memset (scratch,0 ,sizeof (struct xfs_da_args )); +xfs_parent_da_args_init (scratch, tp, pptr, ip, ip->i_ino, parent_name); +return xfs_attr_get_ilocked (scratch); +}
fs/xfs/libxfs/xfs_parent.h +5 -0
@@ -96,4 +96,9 @@ int xfs_parent_from_xattr(struct xfs_mount *mp, unsigned int attr_flags, const void *value, unsigned int valuelen, xfs_ino_t *parent_ino, uint32_t *parent_gen); +/* Repair functions */ +int xfs_parent_lookup (struct xfs_trans *tp,struct xfs_inode *ip, +const struct xfs_name *name,struct xfs_parent_rec *pptr, +struct xfs_da_args *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/dir.c +111 -1
155 lines not on this page · raw message
Re: [PATCH 1/7] xfs: check dirents have parent pointers
Looks good:
PATCH 2/7 xfs: deferred scrub of dirents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based parent pointer check fails, retain those dirents and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/scrub/dir.c | 234 +++++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/readdir.c | 78 ++++++++++++++++ fs/xfs/scrub/readdir.h | 3 + fs/xfs/scrub/trace.h | 34 +++++++ 4 files changed, 346 insertions(+), 3 deletions(-)
fs/xfs/scrub/dir.c +231 -3
@@ -24,6 +24,10 @@#include "scrub/readdir.h" #include "scrub/health.h" #include "scrub/repair.h" +#include "scrub/trace.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" +#include "scrub/xfblob.h" /* Set us up to scrub directories. */ int
@@ -43,12 +47,37 @@ xchk_setup_directory(/* Directories */ +/* Deferred directory entry that we saved for later. */ +struct xchk_dirent { +/* Cookie for retrieval of the dirent name. */ +xfblob_cookie name_cookie; + +/* Child inode number. */ +xfs_ino_t ino; + +/* Length of the pptr name. */ +uint8_t namelen; +}; +struct xchk_dir {struct xfs_scrub *sc;/* information for parent pointer validation. */ struct xfs_parent_rec pptr_rec;struct xfs_da_args pptr_args; + +/* Fixed-size array of xchk_dirent structures. */ +struct xfarray *dir_entries; + +/* Blobs containing dirent names. */ +struct xfblob *dir_names; + +/* If we've cycled the ILOCK, we must revalidate deferred dirents. */ +bool need_revalidate; + +/* Name buffer for dirent revalidation. */ +struct xfs_name xname; +uint8_t namebuf[MAXNAMELEN ]; };/* Scrub a directory entry. */
@@ -148,8 +177,26 @@ xchk_dir_check_pptr_fast(/* Try to lock the inode. */ lockmode =xchk_dir_lock_child (sc, ip);if (!lockmode) { -xchk_set_incomplete (sc); -return -ECANCELED ; +struct xchk_dirent save_de = { + .namelen = name->len, + .ino = ip->i_ino, + }; + +/* Couldn't lock the inode, so save the dirent for later. */ +trace_xchk_dir_defer (sc->ip, name, ip->i_ino); + + error =xfblob_storename (sd->dir_names, &save_de.name_cookie, + name); +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + + error =xfarray_append (sd->dir_entries, &save_de); +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + +return 0 ; } error =xchk_dir_parent_pointer (sd, name, ip);
@@ -865,6 +912,142 @@ xchk_directory_blocks(return error; } +/* +* Revalidate a dirent that we collected in the past but couldn't check because +* of lock contention. Returns 0 if the dirent is still valid, -ENOENT if it +* has gone away on us, or a negative errno. +*/ +STATIC int +xchk_dir_revalidate_dirent ( +struct xchk_dir *sd, +const struct xfs_name *xname, +xfs_ino_t ino) +{ +struct xfs_scrub *sc = sd->sc; +xfs_ino_t child_ino; +int error; + +/* +* Look up the directory entry. If we get -ENOENT, the directory entry +* went away and there's nothing to revalidate. Return any other +* error. +*/ + error =xchk_dir_lookup (sc, sc->ip, xname, &child_ino); +if (error) +return error; + +/* The inode number changed, nothing to revalidate. */ +if (ino != child_ino) +return -ENOENT ; + +return 0 ; +} + +/* +* Check a directory entry's parent pointers the slow way, which means we cycle +* locks a bunch and put up with revalidation until we get it done. +*/ +STATIC int +xchk_dir_slow_dirent ( +struct xchk_dir *sd, +struct xchk_dirent *dirent, +const struct xfs_name *xname) +{ +struct xfs_scrub *sc = sd->sc; +struct xfs_inode *ip; +unsigned int lockmode; +int error; + +/* Check that the deferred dirent still exists. */ +if (sd->need_revalidate) { + error =xchk_dir_revalidate_dirent (sd, xname, dirent->ino); +if (error == -ENOENT ) +return 0 ; +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + } + + error =xchk_iget (sc, dirent->ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; + +/* +* If we can grab both IOLOCK and ILOCK of the alleged child, we can +* proceed with the validation. +*/ + lockmode =xchk_dir_lock_child (sc, ip); +if (lockmode) { +trace_xchk_dir_slowpath (sc->ip, xname, ip->i_ino); + goto check_pptr; + } + +/* +* We couldn't lock the child file. Drop all the locks and try to +* get them again, one at a time. +*/ +xchk_iunlock (sc, sc->ilock_flags); + sd->need_revalidate = true; + +trace_xchk_dir_ultraslowpath (sc->ip, xname, ip->i_ino); + + error =xchk_dir_trylock_for_pptrs (sc, ip, &lockmode); +if (error) + goto out_rele; + +/* Revalidate, since we just cycled the locks. */ + error =xchk_dir_revalidate_dirent (sd, xname, dirent->ino); +if (error == -ENOENT ) { + error =0 ; + goto out_unlock; + } +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , &error)) + goto out_unlock; + +check_pptr: + error =xchk_dir_parent_pointer (sd, xname, ip); +out_unlock: +xfs_iunlock (ip, lockmode); +out_rele: +xchk_irele (sc, ip); +return error; +} + +/* Check all the dirents that we deferred the first time around. */ +STATIC int +xchk_dir_finish_slow_dirents ( +struct xchk_dir *sd) +{ +xfarray_idx_t array_cur; +int error; + + foreach_xfarray_idx(sd->dir_entries, array_cur) { +struct xchk_dirent dirent; + +if (sd->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) +return 0 ; + + error =xfarray_load (sd->dir_entries, array_cur, &dirent); +if (error) +return error; + + error =xfblob_loadname (sd->dir_names, dirent.name_cookie, + &sd->xname, dirent.namelen); +if (error) +return error; + + error =xchk_dir_slow_dirent (sd, &dirent, &sd->xname); +if (error) +return error; + } + +return 0 ; +} +/* Scrub a whole directory. */ int xchk_directory(
@@ -907,11 +1090,56 @@ xchk_directory(if (!sd)return -ENOMEM ; sd->sc = sc; + sd->xname.name = sd->namebuf; + +if (xfs_has_parent (sc->mp)) { +char *descr; + +/* +* Set up some staging memory for dirents that we can't check +* due to locking contention. +*/ + descr =xchk_xfile_ino_descr (sc,"slow directory entries" ); + error =xfarray_create (descr,0 ,sizeof (struct xchk_dirent ), + &sd->dir_entries); +kfree (descr); +if (error) + goto out_sd; + + descr =xchk_xfile_ino_descr (sc,"slow directory entry names" ); + error =xfblob_create (descr, &sd->dir_names); +kfree (descr); +if (error) + goto out_entries; + }/* Look up every name in this directory by hash. */ error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor, sd); +if (error == -ECANCELED ) + error =0 ; +if (error) + goto out_names; + +if (xfs_has_parent (sc->mp)) { + error =xchk_dir_finish_slow_dirents (sd); +if (error == -ETIMEDOUT ) { +/* Couldn't grab a lock, scrub was marked incomplete */ + error =0 ; + goto out_names; + } +if (error) + goto out_names; + } + +out_names: +if (sd->dir_names) +xfblob_destroy (sd->dir_names); +out_entries: +if (sd->dir_entries) +xfarray_destroy (sd->dir_entries); +out_sd:kvfree (sd); -if (error && error != -ECANCELED ) +if (error)return error;/* If the dir is clean, it is clearly not zapped. */
fs/xfs/scrub/readdir.c +78 -0
87 lines not on this page · raw message
fs/xfs/scrub/readdir.h +3 -0
@@ -16,4 +16,7 @@ int xchk_dir_walk(struct xfs_scrub *sc, struct xfs_inode *dp,int xchk_dir_lookup (struct xfs_scrub *sc,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t *ino); +int xchk_dir_trylock_for_pptrs (struct xfs_scrub *sc,struct xfs_inode *ip, +unsigned int *lockmode); +#endif /* __XFS_SCRUB_READDIR_H__ */
fs/xfs/scrub/trace.h +34 -0
@@ -1511,6 +1511,40 @@ DEFINE_EVENT(xchk_nlinks_diff_class, name, \ TP_ARGS(mp, ip, live))DEFINE_SCRUB_NLINKS_DIFF_EVENT (xchk_nlinks_compare_inode); +DECLARE_EVENT_CLASS (xchk_pptr_class, + TP_PROTO(struct xfs_inode *ip, const struct xfs_name *name, + xfs_ino_t far_ino), +TP_ARGS (ip, name, far_ino), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned int, namelen) + __dynamic_array(char, name, name->len) +__field (xfs_ino_t, far_ino) + ), +TP_fast_assign ( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->namelen = name->len; + memcpy(__get_str(name), name, name->len); + __entry->far_ino = far_ino; + ), +TP_printk ("dev %d:%d ino 0x%llx name '%.*s' far_ino 0x%llx" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->namelen, +__get_str (name), + __entry->far_ino) +) +#define DEFINE_XCHK_PPTR_EVENT (name) \ +DEFINE_EVENT(xchk_pptr_class, name, \ + TP_PROTO(struct xfs_inode *ip, const struct xfs_name *name, \ + xfs_ino_t far_ino), \ + TP_ARGS(ip, name, far_ino)) +DEFINE_XCHK_PPTR_EVENT (xchk_dir_defer); +DEFINE_XCHK_PPTR_EVENT (xchk_dir_slowpath); +DEFINE_XCHK_PPTR_EVENT (xchk_dir_ultraslowpath); +/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
Re: [PATCH 2/7] xfs: deferred scrub of dirents
Looks good:
PATCH 3/7 xfs: scrub parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Actually check parent pointers now.
--- fs/xfs/scrub/parent.c | 367 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 367 insertions(+)
fs/xfs/scrub/parent.c +367 -0
@@ -15,11 +15,15 @@#include "xfs_icache.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/readdir.h" #include "scrub/tempfile.h" #include "scrub/repair.h" +#include "scrub/listxattr.h" +#include "scrub/trace.h" /* Set us up to scrub parents. */ int
@@ -197,6 +201,366 @@ xchk_parent_validate(return error; } +/* +* Checking of Parent Pointers +* =========================== +* +* On filesystems with directory parent pointers, we check the referential +* integrity by visiting each parent pointer of a child file and checking that +* the directory referenced by the pointer actually has a dirent pointing +* forward to the child file. +*/ + +struct xchk_pptrs { +struct xfs_scrub *sc; + +/* How many parent pointers did we find at the end? */ +unsigned long long pptrs_found; + +/* Parent of this directory. */ +xfs_ino_t parent_ino; +}; + +/* Does this parent pointer match the dotdot entry? */ +STATIC int +xchk_parent_scan_dotdot ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +void *priv) +{ +struct xchk_pptrs *pp = priv; +xfs_ino_t parent_ino; +int ret; + + ret =xfs_parent_from_xattr (sc->mp, attr_flags, name, namelen, + value, valuelen, &parent_ino,NULL ); +if (ret <0 ) +return ret; + +if (ret ==1 && pp->parent_ino == parent_ino) +return -ECANCELED ; + +return 0 ; +} + +/* Look up the dotdot entry so that we can check it as we walk the pptrs. */ +STATIC int +xchk_parent_pptr_and_dotdot ( +struct xchk_pptrs *pp) +{ +struct xfs_scrub *sc = pp->sc; +int error; + +/* Look up '..' */ + error =xchk_dir_lookup (sc, sc->ip, &xfs_name_dotdot, &pp->parent_ino); +if (!xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; +if (!xfs_verify_dir_ino (sc->mp, pp->parent_ino)) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +/* Is this the root dir? Then '..' must point to itself. */ +if (sc->ip == sc->mp->m_rootip) { +if (sc->ip->i_ino != pp->parent_ino) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +/* +* If this is now an unlinked directory, the dotdot value is +* meaningless as long as it points to a valid inode. +*/ +if (VFS_I (sc->ip)->i_nlink ==0 ) +return 0 ; + +if (pp->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) +return 0 ; + +/* Otherwise, walk the pptrs again, and check. */ + error =xchk_xattr_walk (sc, sc->ip, xchk_parent_scan_dotdot, pp); +if (error == -ECANCELED ) { +/* Found a parent pointer that matches dotdot. */ +return 0 ; + } +if (!error || error == -EFSCORRUPTED ) { +/* Found a broken parent pointer or no match. */ +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +return error; +} + +/* +* Try to lock a parent directory for checking dirents. Returns the inode +* flags for the locks we now hold, or zero if we failed. +*/ +STATIC unsigned int +xchk_parent_lock_dir ( +struct xfs_scrub *sc, +struct xfs_inode *dp) +{ +if (!xfs_ilock_nowait (dp,XFS_IOLOCK_SHARED )) +return 0 ; + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_SHARED )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +if (!xfs_need_iread_extents (&dp->i_df)) +return XFS_IOLOCK_SHARED |XFS_ILOCK_SHARED ; + +xfs_iunlock (dp,XFS_ILOCK_SHARED ); + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_EXCL )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +return XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; +} + +/* Check the forward link (dirent) associated with this parent pointer. */ +STATIC int +xchk_parent_dirent ( +struct xchk_pptrs *pp, +const struct xfs_name *xname, +struct xfs_inode *dp) +{ +struct xfs_scrub *sc = pp->sc; +xfs_ino_t child_ino; +int error; + +/* +* Use the name attached to this parent pointer to look up the +* directory entry in the alleged parent. +*/ + error =xchk_dir_lookup (sc, dp, xname, &child_ino); +if (error == -ENOENT ) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* Does the inode number match? */ +if (child_ino != sc->ip->i_ino) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } + +return 0 ; +} + +/* Try to grab a parent directory. */ +STATIC int +xchk_parent_iget ( +struct xchk_pptrs *pp, +const struct xfs_parent_rec *pptr, +struct xfs_inode **dpp) +{ +struct xfs_scrub *sc = pp->sc; +struct xfs_inode *ip; +xfs_ino_t parent_ino =be64_to_cpu (pptr->p_ino); +int error; + +/* Validate inode number. */ + error =xfs_dir_ino_validate (sc->mp, parent_ino); +if (error) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + + error =xchk_iget (sc, parent_ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* The parent must be a directory. */ +if (!S_ISDIR (VFS_I (ip)->i_mode)) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + +/* Validate generation number. */ +if (VFS_I (ip)->i_generation !=be32_to_cpu (pptr->p_gen)) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + + *dpp = ip; +return 0 ; +out_rele: +xchk_irele (sc, ip); +return 0 ; +} + +/* +* Walk an xattr of a file. If this xattr is a parent pointer, follow it up +* to a parent directory and check that the parent has a dirent pointing back +* to us. +*/ +STATIC int +xchk_parent_scan_attr ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +void *priv) +{ +struct xfs_name xname = { + .name = name, + .len = namelen, + }; +struct xchk_pptrs *pp = priv; +struct xfs_inode *dp =NULL ; +const struct xfs_parent_rec *pptr_rec = value; +xfs_ino_t parent_ino; +unsigned int lockmode; +int ret; + + ret =xfs_parent_from_xattr (sc->mp, attr_flags, name, namelen, + value, valuelen, &parent_ino,NULL ); +if (ret <0 ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } +if (ret !=1 ) +return 0 ; + +/* No self-referential parent pointers. */ +if (parent_ino == sc->ip->i_ino) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + + pp->pptrs_found++; + + ret =xchk_parent_iget (pp, pptr_rec, &dp); +if (ret) +return ret; +if (!dp) +return 0 ; + +/* Try to lock the inode. */ + lockmode =xchk_parent_lock_dir (sc, dp); +if (!lockmode) { +xchk_set_incomplete (sc); + ret = -ECANCELED ; + goto out_rele; + } + + ret =xchk_parent_dirent (pp, &xname, dp); +if (ret) + goto out_unlock; + +out_unlock: +xfs_iunlock (dp, lockmode); +out_rele: +xchk_irele (sc, dp); +return ret; +} + +/* +* Compare the number of parent pointers to the link count. For +* non-directories these should be the same. For unlinked directories the +* count should be zero; for linked directories, it should be nonzero. +*/ +STATIC int +xchk_parent_count_pptrs ( +struct xchk_pptrs *pp) +{ +struct xfs_scrub *sc = pp->sc; + +if (S_ISDIR (VFS_I (sc->ip)->i_mode)) { +if (sc->ip == sc->mp->m_rootip) + pp->pptrs_found++; + +if (VFS_I (sc->ip)->i_nlink ==0 && pp->pptrs_found >0 ) +xchk_ino_set_corrupt (sc, sc->ip->i_ino); +else if (VFS_I (sc->ip)->i_nlink >0 && + pp->pptrs_found ==0 ) +xchk_ino_set_corrupt (sc, sc->ip->i_ino); + }else { +if (VFS_I (sc->ip)->i_nlink != pp->pptrs_found) +xchk_ino_set_corrupt (sc, sc->ip->i_ino); + } + +return 0 ; +} + +/* Check parent pointers of a file. */ +STATIC int +xchk_parent_pptr ( +struct xfs_scrub *sc) +{ +struct xchk_pptrs *pp; +int error; + + pp =kvzalloc (sizeof (struct xchk_pptrs ),XCHK_GFP_FLAGS ); +if (!pp) +return -ENOMEM ; + pp->sc = sc; + + error =xchk_xattr_walk (sc, sc->ip, xchk_parent_scan_attr, pp); +if (error == -ECANCELED ) { + error =0 ; + goto out_pp; + } +if (error) + goto out_pp; + +if (pp->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) + goto out_pp; + +/* +* For subdirectories, make sure the dotdot entry references the same +* inode as the parent pointers. +* +* If we're scanning a /consistent/ directory, there should only be +* one parent pointer, and it should point to the same directory as +* the dotdot entry. +* +* However, a corrupt directory tree might feature a subdirectory with +* multiple parents. The directory loop scanner is responsible for +* correcting that kind of problem, so for now we only validate that +* the dotdot entry matches /one/ of the parents. +*/ +if (S_ISDIR (VFS_I (sc->ip)->i_mode)) { + error =xchk_parent_pptr_and_dotdot (pp); +if (error) + goto out_pp; + } + +if (pp->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) + goto out_pp; + +/* +* Complain if the number of parent pointers doesn't match the link +* count. This could be a sign of missing parent pointers (or an +* incorrect link count). +*/ + error =xchk_parent_count_pptrs (pp); +if (error) + goto out_pp; + +out_pp: +kvfree (pp); +return error; +} +/* Scrub a parent pointer. */ int xchk_parent(
@@ -206,6 +570,9 @@ xchk_parent(xfs_ino_t parent_ino;int error =0 ; +if (xfs_has_parent (mp)) +return xchk_parent_pptr (sc); +/* * If we're a directory, check that the '..' link points up to * a directory that has one entry pointing to us.
Re: [PATCH 3/7] xfs: scrub parent pointers
Looks good:
PATCH 4/7 xfs: deferred scrub of parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based dirent check fails, retain those parent pointers and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/Makefile | 2 fs/xfs/scrub/parent.c | 264 ++++++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/trace.h | 3 + 3 files changed, 261 insertions(+), 8 deletions(-)
fs/xfs/Makefile +1 -1
@@ -177,6 +177,7 @@ xfs-y += $(addprefix scrub/, \ scrub.o \ symlink.o \ xfarray.o \ + xfblob.o \ xfile.o \ )
@@ -218,7 +219,6 @@ xfs-y += $(addprefix scrub/, \ rmap_repair.o \ symlink_repair.o \ tempfile.o \ - xfblob.o \ ) xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \
fs/xfs/scrub/parent.c +257 -7
333 lines not on this page · raw message
fs/xfs/scrub/trace.h +3 -0
@@ -1544,6 +1544,9 @@ DEFINE_EVENT(xchk_pptr_class, name, \DEFINE_XCHK_PPTR_EVENT (xchk_dir_defer);DEFINE_XCHK_PPTR_EVENT (xchk_dir_slowpath);DEFINE_XCHK_PPTR_EVENT (xchk_dir_ultraslowpath); +DEFINE_XCHK_PPTR_EVENT (xchk_parent_defer); +DEFINE_XCHK_PPTR_EVENT (xchk_parent_slowpath); +DEFINE_XCHK_PPTR_EVENT (xchk_parent_ultraslowpath);/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
Re: [PATCH 4/7] xfs: deferred scrub of parent pointers
Looks good:
PATCH 5/7 xfs: walk directory parent pointers to determine backref count
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the filesystem has parent pointers enabled, walk the parent pointers of subdirectories to determine the true backref count. In theory each subdir should have a single parent reachable via dotdot, but in the case of (corrupt) subdirs with multiple parents, we need to keep the link counts high enough that the directory loop detector will be able to correct the multiple parents problems.
--- fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/nlinks.c | 82 +++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/nlinks_repair.c | 2 + fs/xfs/scrub/parent.c | 61 +++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.c | 1 + fs/xfs/scrub/trace.h | 28 ++++++++++++++ 6 files changed, 174 insertions(+), 1 deletion(-)
fs/xfs/scrub/common.h +1 -0
@@ -212,6 +212,7 @@ static inline bool xchk_skip_xref(struct xfs_scrub_metadata *sm) }bool xchk_dir_looks_zapped (struct xfs_inode *dp); +bool xchk_pptr_looks_zapped (struct xfs_inode *ip);#ifdef CONFIG_XFS_ONLINE_REPAIR /* Decide if a repair is required. */
fs/xfs/scrub/nlinks.c +81 -1
117 lines not on this page · raw message
fs/xfs/scrub/nlinks_repair.c +2 -0
@@ -18,6 +18,8 @@#include "xfs_ialloc.h" #include "xfs_sb.h" #include "xfs_ag.h" +#include "xfs_dir2.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/repair.h"
fs/xfs/scrub/parent.c +61 -0
64 lines not on this page · raw message
fs/xfs/scrub/trace.c +1 -0
@@ -19,6 +19,7 @@#include "xfs_da_format.h" #include "xfs_dir2.h" #include "xfs_rmap.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/xfile.h" #include "scrub/xfarray.h"
fs/xfs/scrub/trace.h +28 -0
@@ -26,6 +26,7 @@ struct xchk_iscan;struct xchk_nlink ;struct xchk_fscounters ;struct xfs_rmap_update_params ; +struct xfs_parent_rec ;/* * ftrace's __print_symbolic requires that all enum values be wrapped in the
@@ -1363,6 +1364,33 @@ TRACE_EVENT(xchk_nlinks_collect_dirent, __get_str(name)) ); +TRACE_EVENT (xchk_nlinks_collect_pptr, + TP_PROTO(struct xfs_mount *mp, struct xfs_inode *dp, + const struct xfs_name *name, + const struct xfs_parent_rec *pptr), +TP_ARGS (mp, dp, name, pptr), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, dir) + __field(xfs_ino_t, ino) + __field(unsigned int, namelen) +__dynamic_array (char, name, name->len) + ), +TP_fast_assign ( + __entry->dev = mp->m_super->s_dev; + __entry->dir = dp->i_ino; + __entry->ino = be64_to_cpu(pptr->p_ino); + __entry->namelen = name->len; + memcpy(__get_str(name), name->name, name->len); + ), +TP_printk ("dev %d:%d dir 0x%llx -> ino 0x%llx name '%.*s'" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->dir, + __entry->ino, + __entry->namelen, +__get_str (name)) +); + TRACE_EVENT(xchk_nlinks_collect_metafile, TP_PROTO(struct xfs_mount *mp, xfs_ino_t ino), TP_ARGS(mp, ino),
Re: [PATCH 5/7] xfs: walk directory parent pointers to determine backref count
Looks good:
PATCH 6/7 xfs: check parent pointer xattrs when scrubbing
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check parent pointer xattrs as part of scrubbing xattrs.
--- fs/xfs/scrub/attr.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/xfs/scrub/attr.c +8 -0
@@ -17,6 +17,7 @@#include "xfs_attr.h" #include "xfs_attr_leaf.h" #include "xfs_attr_sf.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/dabtree.h"
@@ -208,6 +209,13 @@ xchk_xattr_actor(return -ECANCELED ; } +/* Check parent pointer record. */ +if ((attr_flags &XFS_ATTR_PARENT ) && + !xfs_parent_valuecheck (sc->mp, value, valuelen)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno); +return -ECANCELED ; + } +/* * Local and shortform xattr values are stored in the attr leaf block, * so we don't need to retrieve the value from a remote block to detect
Re: [PATCH 6/7] xfs: check parent pointer xattrs when scrubbing
Looks good:
PATCH 7/7 xfs: salvage parent pointers when rebuilding xattr structures
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When we're salvaging extended attributes, make sure we validate the ones that claim to be parent pointers before adding them to the salvage pile.
--- fs/xfs/scrub/attr_repair.c | 34 +++++++++++++++++++++++++--------- fs/xfs/scrub/trace.h | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 9 deletions(-)
fs/xfs/scrub/attr_repair.c +25 -9
@@ -28,6 +28,7 @@#include "xfs_exchmaps.h" #include "xfs_exchrange.h" #include "xfs_acl.h" +#include "xfs_parent.h" #include "scrub/xfs_scrub.h" #include "scrub/scrub.h" #include "scrub/common.h"
@@ -127,6 +128,9 @@ xrep_xattr_want_salvage(return false;if (valuelen >XATTR_SIZE_MAX || valuelen <0 )return false; +if (attr_flags &XFS_ATTR_PARENT ) +return xfs_parent_valuecheck (rx->sc->mp, value, valuelen); +return true; }
@@ -154,14 +158,21 @@ xrep_xattr_salvage_key(* Truncate the name to the first character that would trip namecheck. * If we no longer have a name after that, ignore this attribute. */ -while (i < namelen && name[i] !=0 ) - i++; -if (i ==0 ) -return 0 ; - key.namelen = i; +if (flags &XFS_ATTR_PARENT ) { + key.namelen = namelen; -trace_xrep_xattr_salvage_rec (rx->sc->ip, flags, name, key.namelen, - valuelen); +trace_xrep_xattr_salvage_pptr (rx->sc->ip, flags, name, + key.namelen, value, valuelen); + }else { +while (i < namelen && name[i] !=0 ) + i++; +if (i ==0 ) +return 0 ; + key.namelen = i; + +trace_xrep_xattr_salvage_rec (rx->sc->ip, flags, name, + key.namelen, valuelen); + } error =xfblob_store (rx->xattr_blobs, &key.name_cookie, name, key.namelen);
@@ -596,8 +607,13 @@ xrep_xattr_insert_rec( ab->name[key->namelen] =0 ; -trace_xrep_xattr_insert_rec (rx->sc->tempip, key->flags, ab->name, - key->namelen, key->valuelen); +if (key->flags &XFS_ATTR_PARENT ) +trace_xrep_xattr_insert_pptr (rx->sc->tempip, key->flags, + ab->name, key->namelen, ab->value, + key->valuelen); +else +trace_xrep_xattr_insert_rec (rx->sc->tempip, key->flags, + ab->name, key->namelen, key->valuelen);/* * xfs_attr_set creates and commits its own transaction. If the attr
fs/xfs/scrub/trace.h +38 -0
@@ -2540,6 +2540,44 @@ DEFINE_EVENT(xrep_xattr_salvage_class, name, \DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_xattr_salvage_rec);DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_xattr_insert_rec); +DECLARE_EVENT_CLASS (xrep_pptr_salvage_class, + TP_PROTO(struct xfs_inode *ip, unsigned int flags, const void *name, + unsigned int namelen, const void *value, unsigned int valuelen), +TP_ARGS (ip, flags, name, namelen, value, valuelen), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(xfs_ino_t, parent_ino) + __field(unsigned int, parent_gen) + __field(unsigned int, namelen) +__dynamic_array (char, name, namelen) + ), +TP_fast_assign ( + const struct xfs_parent_rec *rec = value; + + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->parent_ino = be64_to_cpu(rec->p_ino); + __entry->parent_gen = be32_to_cpu(rec->p_gen); + __entry->namelen = namelen; + memcpy(__get_str(name), name, namelen); + ), +TP_printk ("dev %d:%d ino 0x%llx parent_ino 0x%llx parent_gen 0x%x name '%.*s'" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->parent_ino, + __entry->parent_gen, + __entry->namelen, +__get_str (name)) +) +#define DEFINE_XREP_PPTR_SALVAGE_EVENT (name) \ +DEFINE_EVENT(xrep_pptr_salvage_class, name, \ + TP_PROTO(struct xfs_inode *ip, unsigned int flags, const void *name, \ + unsigned int namelen, const void *value, unsigned int valuelen), \ + TP_ARGS(ip, flags, name, namelen, value, valuelen)) +DEFINE_XREP_PPTR_SALVAGE_EVENT (xrep_xattr_salvage_pptr); +DEFINE_XREP_PPTR_SALVAGE_EVENT (xrep_xattr_insert_pptr); + TRACE_EVENT(xrep_xattr_class, TP_PROTO(struct xfs_inode *ip, struct xfs_inode *arg_ip), TP_ARGS(ip, arg_ip),
Re: [PATCH 7/7] xfs: salvage parent pointers when rebuilding xattr structures
Looks good:
PATCH 7/9 xfs: online repair for parent pointers
In reply to Darrick J. WongHi all,
This series implements online repair for directory parent pointer metadata. The checking half is fairly straightforward -- for each outgoing directory link (forward or backwards), grab the inode at the other end, and confirm that there's a corresponding link. If we can't grab an inode or lock it, we'll save that link for a slower loop that cycles all the locks, confirms the continued existence of the link, and rechecks the link if it's actually still there.
Repairs are a bit more involved -- for directories, we walk the entire filesystem to rebuild the dirents from parent pointer information. Parent pointer repairs do the same walk but rebuild the pptrs from the dirent information, but with the added twist that it duplicates all the xattrs so that it can use the atomic extent swapping code to commit the repairs atomically.
This introduces an added twist to the xattr repair code -- we use dirent hooks to detect a colliding update to the pptr data while we're not holding the ILOCKs; if one is detected, we restart the xattr salvaging process but this time hold all the ILOCKs until the end of the scan.
For offline repair, the phase6 directory connectivity scan generates an index of all the expected parent pointers in the filesystem. Then it walks each file and compares the parent pointers attached to that file against the index generated, and resyncs the results as necessary.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-pptrs
---
Commits in this patchset:
* xfs: add xattr setname and removename functions for internal users
* xfs: add raw parent pointer apis to support repair
* xfs: repair directories by scanning directory parent pointers
* xfs: implement live updates for directory repairs
* xfs: replay unlocked parent pointer updates that accrue during xattr repair
* xfs: repair directory parent pointers by scanning for dirents
* xfs: implement live updates for parent pointer repairs
* xfs: remove pointless unlocked assertion
* xfs: split xfs_bmap_add_attrfork into two pieces
* xfs: add a per-leaf block callback to xchk_xattr_walk
* xfs: actually rebuild the parent pointer xattrs
* xfs: adapt the orphanage code to handle parent pointers
* xfs: repair link count of nondirectories after rebuilding parent pointers
* xfs: inode repair should ensure there's an attr fork to store parent pointers
--- fs/xfs/libxfs/xfs_attr.c | 230 +++++++ fs/xfs/libxfs/xfs_attr.h | 4 fs/xfs/libxfs/xfs_bmap.c | 38 - fs/xfs/libxfs/xfs_bmap.h | 3 fs/xfs/libxfs/xfs_dir2.c | 2 fs/xfs/libxfs/xfs_dir2.h | 2 fs/xfs/libxfs/xfs_parent.c | 64 ++ fs/xfs/libxfs/xfs_parent.h | 6 fs/xfs/scrub/attr.c | 2 fs/xfs/scrub/attr_repair.c | 459 +++++++++++++++ fs/xfs/scrub/attr_repair.h | 4 fs/xfs/scrub/dir_repair.c | 564 +++++++++++++++++- fs/xfs/scrub/findparent.c | 12 fs/xfs/scrub/findparent.h | 10 fs/xfs/scrub/inode_repair.c | 41 + fs/xfs/scrub/listxattr.c | 10 fs/xfs/scrub/listxattr.h | 4 fs/xfs/scrub/nlinks.c | 3 fs/xfs/scrub/orphanage.c | 38 + fs/xfs/scrub/orphanage.h | 3 fs/xfs/scrub/parent.c | 7 fs/xfs/scrub/parent_repair.c | 1301 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.h | 115 ++++ 24 files changed, 2816 insertions(+), 108 deletions(-)