xfs: Add support for multi rtgroup shrink+removal
Hi all,
This series adds online support for shrinking and removing multiple realtime groups in XFS.
Currently, realtime group shrink/removal operations are not available. This series extends the infrastructure to allow online shrinking/removal of multiple rtgroups in a single operation.
The work is organized into three sub-patchsets:
1. Kernel changes
- Core filesystem changes enabling multi-rtgroup shrink/removal.
2. xfsprogs updates
- Changes to xfs_growfs allow the shrink requests to be passed to the kernel.
3. fstests additions
- Test coverage validating grow, shrink and removal of multiple rtgroups.
Each patch includes details describing the implementation and design considerations.
Comments and reviews are greatly appreciated.
cover xfs: Add support to shrink multiple empty rtgroups
From: "Nirjhar Roy (IBM)" <[email protected]>
This work is based on a previous RFC[1] by Gao Xiang and various ideas proposed by Dave Chinner in the RFC[1].
The patch begins with the re-introduction of some of the data structures that were removed, some code refactoring and finally the patch that implements the multi rtgroup shrink design. We can remove only those rtgroups which are empty.
For non-empty rtgroups the rtextents can migrated to other rtgroups (assuming the required amount of space is available).
I am working on the patch series [2](by Darrick and
Dave) for completion of the data block migration from the end of the filesystem for emptying an realtime group. That will be a future work after this.
The final patch has all the details including the definition of the terminologies and the overall design.
The design is quite similar to the design of data AG removal implemented in the patch series[3] that I have posted earlier.
The reason for keeping [3] on hold and posting this patch series is that (based on the discussion in [5], [6]), realtime devices won't have any metadata/inodes and migrating data from the end of realtime devices will be simpler. On the other hand there are challenges in moving metadata from regular AGs especially inodes.
Please note that I have added RBs from Darrick in patch 1 which was given in [4].
Instructions to test this patch:
$ Apply the patch for xfsprogs sent with this series and install it. $ mkfs.xfs -f -m metadir=1 -r rtdev=/dev/loop2,extsize=4096,rgcount=4,size=1G \ -d size=1G /dev/loop1 $ mount -o rtdev=/dev/loop2 /dev/loop1 /mnt/scratch $ # shrink by 1.5 rtgroups $ xfs_growfs -R $(( 65536 * 2 + 32768 )) /mnt1/scratch
I have also sent the tests.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-xfs/173568777852.2709794.6356870909327619205.stgit@frogsfrogsfrogs/
[3] https://lore.kernel.org/linux-xfs/[email protected]/
[4] https://lore.kernel.org/all/20250729202632.GF2672049@frogsfrogsfrogs/
[5] https://lore.kernel.org/linux-xfs/[email protected]/
[6] https://lore.kernel.org/linux-xfs/[email protected]/
Nirjhar Roy (IBM) (4):
xfs: Re-introduce xg_active_wq field in struct xfs_group
xfs: Introduce xfs_rtginodes_ensure_all()
xfs: Add a new state for shrinking
xfs: Add support to shrink multiple empty realtime groups
fs/xfs/libxfs/xfs_group.c | 18 +- fs/xfs/libxfs/xfs_group.h | 4 + fs/xfs/libxfs/xfs_rtgroup.c | 105 ++++- fs/xfs/libxfs/xfs_rtgroup.h | 31 ++ fs/xfs/xfs_buf_item_recover.c | 25 +- fs/xfs/xfs_extent_busy.c | 30 ++ fs/xfs/xfs_extent_busy.h | 2 + fs/xfs/xfs_inode.c | 8 +- fs/xfs/xfs_mount.h | 3 + fs/xfs/xfs_rtalloc.c | 824 +++++++++++++++++++++++++++++++++- fs/xfs/xfs_trans.c | 1 - 11 files changed, 1023 insertions(+), 28 deletions(-)
--
2.43.5
RFC 1/4 xfs: Re-introduce xg_active_wq field in struct xfs_group
From: "Nirjhar Roy (IBM)" <[email protected]>
pag_active_wq was removed in
commit 9943b4573290
("xfs: remove the unused pag_active_wq field in struct xfs_perag")
because it was not waited upon. Re-introducing this in struct xfs_group. This patch also replaces atomic_dec() in xfs_group_rele() with
if (atomic_dec_and_test(&xg->xg_active_ref))
wake_up(&xg->xg_active_wq);
The reason for this change is that the online shrink code will wait for all the active references to come down to zero before actually starting the shrink process (only if the number of rtextents that we are trying to remove is worth 1 or more rtgroups).
--- fs/xfs/libxfs/xfs_group.c | 4 +++- fs/xfs/libxfs/xfs_group.h | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_group.c +3 -1
@@ -147,7 +147,8 @@ xfs_group_rele( struct xfs_group *xg) {trace_xfs_group_rele (xg, _RET_IP_); -atomic_dec (&xg->xg_active_ref); +if (atomic_dec_and_test (&xg->xg_active_ref)) +wake_up (&xg->xg_active_wq); }void
@@ -202,6 +203,7 @@ xfs_group_insert(xfs_defer_drain_init (&xg->xg_intents_drain);/* Active ref owned by mount indicates group is online. */ +init_waitqueue_head (&xg->xg_active_wq);atomic_set (&xg->xg_active_ref,1 ); error =xa_insert (&mp->m_groups[type].xa, index, xg,GFP_KERNEL );
fs/xfs/libxfs/xfs_group.h +2 -0
@@ -11,6 +11,8 @@ struct xfs_group {enum xfs_group_type xg_type;atomic_t xg_ref;/* passive reference count */ atomic_t xg_active_ref;/* active reference count */ +/* woken up when xg_active_ref falls to zero */ +wait_queue_head_t xg_active_wq;/* Precalculated geometry info */ uint32_t xg_block_count;/* max usable gbno */
Signature · 1 line
2.43.5
RFC 3/4 xfs: Add a new state for shrinking
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
Introduce a new state for shrinking. This will be used in the upcoming patch that implements online shrinking of realtime XFS.
--- fs/xfs/xfs_mount.h | 3 +++ 1 file changed, 3 insertions(+)
fs/xfs/xfs_mount.h +3 -0
@@ -577,6 +577,8 @@ __XFS_HAS_FEAT(nouuid, NOUUID)#define XFS_OPSTATE_WARNED_ZONED 19/* (Zoned) GC is in progress */ #define XFS_OPSTATE_ZONEGC_RUNNING 20 +/* filesystem is shrinking */ +#define XFS_OPSTATE_SHRINKING 21#define __XFS_IS_OPSTATE (name,NAME ) \ static inline bool xfs_is_ ## name (struct xfs_mount *mp) \
@@ -599,6 +601,7 @@ __XFS_IS_OPSTATE(inode32, INODE32) __XFS_IS_OPSTATE(readonly, READONLY) __XFS_IS_OPSTATE(inodegc_enabled, INODEGC_ENABLED) __XFS_IS_OPSTATE(blockgc_enabled, BLOCKGC_ENABLED) +__XFS_IS_OPSTATE(shrinking, SHRINKING)#ifdef CONFIG_XFS_QUOTA __XFS_IS_OPSTATE(quotacheck_running, QUOTACHECK_RUNNING)__XFS_IS_OPSTATE (resuming_quotaon,RESUMING_QUOTAON )
Signature · 1 line
2.43.5
RFC 2/4 xfs: Introduce xfs_rtginodes_ensure_all()
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
This is just a wrapper to load and initialize all the realtime metadata inodes. This will be used in the later patches for realtime xfs shrink. No functional changes.
--- fs/xfs/libxfs/xfs_rtgroup.h | 2 ++ fs/xfs/xfs_rtalloc.c | 25 +++++++++++++++++++------ 2 files changed, 21 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_rtgroup.h +2 -0
@@ -109,6 +109,8 @@ static inline struct xfs_inode *rtg_refcount(const struct xfs_rtgroup *rtg)return rtg->rtg_inodes[XFS_RTGI_REFCOUNT ]; } +int xfs_rtginodes_ensure_all (struct xfs_rtgroup *rtg); +/* Passive rtgroup references */ static inline struct xfs_rtgroup * xfs_rtgroup_get(
fs/xfs/xfs_rtalloc.c +19 -6
@@ -1192,18 +1192,15 @@ xfs_growfs_rtg(xfs_extlen_t bmblocks;xfs_fileoff_t bmbno;struct xfs_rtgroup *rtg; -unsigned int i;int error; rtg =xfs_rtgroup_grab (mp, rgno);if (!rtg)return -EINVAL ; -for (i =0 ; i <XFS_RTGI_MAX ; i++) { - error =xfs_rtginode_ensure (rtg, i); -if (error) - goto out_rele; - } + error =xfs_rtginodes_ensure_all (rtg); +if (error) + goto out_rele;if (xfs_has_zoned (mp)) { error =xfs_growfs_rt_zoned (rtg, nrblocks);
@@ -1300,6 +1297,22 @@ xfs_growfs_check_rtgeom(return -EINVAL ; } +int +xfs_rtginodes_ensure_all (struct xfs_rtgroup *rtg) +{ +int i =0 ; +int error =0 ; + +ASSERT (rtg); + +for (i =0 ; i <XFS_RTGI_MAX ; i++) { + error =xfs_rtginode_ensure (rtg, i); +if (error) +break ; + } +return error; +} +/* * Compute the new number of rt groups and ensure that /rtgroups exists. *
Signature · 1 line
2.43.5
RFC 4/4 xfs: Add support to shrink multiple empty realtime groups
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
This patch is based on a previous RFC[1] by Gao Xiang and various ideas proposed by Dave Chinner in the RFC[1].
This patch adds the functionality to shrink the filesystem beyond 1 rtgroup. We can remove only empty rtgroups in order to prevent loss of data. Before I summarize the overall steps of the shrink process, I would like to introduce some of the terminologies:
1. Empty rtgroup - An rtgroup with no allocated space. Removal of this rtgroup will not result in any data loss.
2. Active/Online rtgroup - Online rtgroup and active rtgroup will be used interchangebly. An rtgroup is active or online when all the regular operations can be done on it. When we mount a filesystem, all the rtgroups are by default online/active. In terms of implementation, an online rtgroup will have number of active references greater than 0 (the default value is 1 i.e, an rtgroup by default is online/active).
3. rtgroup offlining/deactivation - rtgroup offlining and rtgroup deactivation will be used interchangebly. An rtgroup is said to be offlined/deactivated when no new high level operation can be started on the rtgroup. This is implemented with the help of active references. When the active reference count of an rtgroup is 0, the rtgroup is said to be deactivated.
No new active reference can be taken if the present active reference count is 0. This way a barrier is formed from preventing new high level operations to get started on an already offlined rtgroup.
4. Reactivating an rtgroup - If we try to remove an offlined rtgroup but for some reason, we can't, then we reactivate the rtgroup i.e, the rtgroup will once more be in an usable state i.e, the active reference count will be set to 1. All the high level operations can now be performed on this rtgroup. In terms of implementation, in order to activate an rtgroup, we atomically set the active reference count to 1.
5. rtgroup removal - This means that an rtgroup no longer exists in the filesystem.
6. New tail rtgroup - This refers to the last rtgroup that will be formed after the removal of 1 or more rtgroups. For example, if there are 4 rtgroups, each with 32 rtextents, then there are total of 4 * 32 = 128 rtextents. Now, if we remove 40 rtextents, rtgroup 3(indexed at 0) will be completely removed (32 rtextents) and from AG 2, we will remove 8 rtextents. So rtgroup 2 will be the new tail rtgroup with 24 rtextents.
7. Old tail rtgroup - This is the last rtgroup before the start of the shrink process. If the number of rtextents removed is less than the last rtgroup's size, then the old tail rtgroup will be the same as the new tail rtgroup.
8. rtgroup stabilization - This simply means that the in-memory contents are synched to the disk.
The overall steps for the removal of rtgroup(s) are as follows:
PHASE 1: Preparing the rtgroups for removal
1. Deactivate the rtgroups to be removed completely - This is done by the function xfs_shrinkfs_rt_deactivate_rtgroups(). The steps to deactivate an rtgrpoup are as follows(function is xfs_rtgroup_deactivate()):
1.a Manually reserve/reduce from the global rtextent free counters
1 rtgroup worth of rtextents. This is done in order to prevent a race where, some rtgroups have been offlined but the delayed allocator has already promised some bytes and the real rtextent allocation is failing due to the rtgroup(s) being offline.
So shrink operation reserves to the shrink transaction the space to be removed from the incore rtextents and either commits that change to the ondisk rtextents
(shrink succeeds) or gives it back (shrink fails).
1.b Wait for the active reference to come to 0.
This is done so that no other entity is racing while the removal is in progress i.e, no new high level operation can start on that rtgroup while we are trying to remove the rtgroup. rtgroup deactivation will fail if the rtgroup is non-empty at the time of deactivation.
2. Once we have waited for the active references to come down to 0, we make sure that all the pending operations on that rtgroup are completed and the in-core and on-disk structures are in synch i.e, the rtgroup is stabilized on to the disk.
The steps to stablize the rtgroup onto the disk are as follows:
2.a We need to flush and empty the logs and wait for all the pending
I/Os to complete - for this, perform a log force+ail push by calling xfs_ail_push_all_sync(). This also ensures that none of the future logged transactions will refer to these rtgroups during log recovery in case if sudden shutdown/crash happens while we are trying to remove these rtgroups. We also sync the superblock with the disk.
2.b Wait for all the busy extents for the target rtgroups to be
resolved (done by the function xfs_extent_busy_wait_rtgroups())
2.c Flush the xfs_discard_wq workqueue
3. Once the rtgroup is deactivated and stabilized on to the disk, we check if all the target rtgroups are empty, and if not, we fail the shrink process.
PHASE 2: Shrink new tail rtgroup, punch out totally empty rtgroups
4. Once the preparation phase is over, we start the actual removal process. This is done in the function xfs_shrinkfs_rt_remove_rtgroups().
Here, we first remove the rtgroups completely and then update the metadata of the new tail rtgroup. The (new) tail rtgroup metadata is shrunk in the function xfs_shrinkfs_rt_shrink_new_tail_rtgroup().
5. In the end we log the changes and commit the transaction.
Removal of each incore rtgroup structure is done by the function xfs_shrinkfs_rt_remove_rtgroup().
The steps can be outlined as follows:
1. Call xfs_rtginodes_ensure_all() to load all the metadata inodes for the target rtgroup.
2. Truncate the data blocks for all the metadata files for that rtgroup - done by the function xfs_rt_metainodes_truncate().
3. Remove the metadata inodes for the rtgroup - done by the function xfs_rt_metainodes_remove().
4. Erase the group from the xarray.
5. Freeing the intents drain queue - done by the function xfs_defer_drain_free().
6. Freeing busy extents list.
7. Freeing the struct xfs_rtgroup pointer - We assert that all the active and passive references are down to 0.
[1] https://lore.kernel.org/all/[email protected]/
--- fs/xfs/libxfs/xfs_group.c | 14 + fs/xfs/libxfs/xfs_group.h | 2 + fs/xfs/libxfs/xfs_rtgroup.c | 105 ++++- fs/xfs/libxfs/xfs_rtgroup.h | 29 ++ fs/xfs/xfs_buf_item_recover.c | 25 +- fs/xfs/xfs_extent_busy.c | 30 ++ fs/xfs/xfs_extent_busy.h | 2 + fs/xfs/xfs_inode.c | 8 +- fs/xfs/xfs_rtalloc.c | 799 +++++++++++++++++++++++++++++++++- fs/xfs/xfs_trans.c | 1 - 10 files changed, 994 insertions(+), 21 deletions(-)
fs/xfs/libxfs/xfs_group.c +14 -0
@@ -230,3 +230,17 @@ xfs_group_get_by_fsb( {return xfs_group_get (mp,xfs_fsb_to_gno (mp, fsbno, type), type); } + +int +xfs_group_get_active_refcount (struct xfs_group *xg) +{ +ASSERT (xg); +return atomic_read (&xg->xg_active_ref); +} + +int +xfs_group_get_passive_refcount (struct xfs_group *xg) +{ +ASSERT (xg); +return atomic_read (&xg->xg_ref); +}
fs/xfs/libxfs/xfs_group.h +2 -0
@@ -83,6 +83,8 @@ void xfs_group_free(struct xfs_mount *mp, uint32_t index, enum xfs_group_type type, void (*uninit)(struct xfs_group *xg));int xfs_group_insert (struct xfs_mount *mp,struct xfs_group *xg,uint32_t index,enum xfs_group_type ); +int xfs_group_get_active_refcount (struct xfs_group *xg); +int xfs_group_get_passive_refcount (struct xfs_group *xg);#define xfs_group_set_mark (_xg, _mark) \ xa_set_mark(&(_xg)->xg_mount->m_groups[(_xg)->xg_type].xa, \
fs/xfs/libxfs/xfs_rtgroup.c +101 -4
125 lines not on this page · raw message
fs/xfs/libxfs/xfs_rtgroup.h +29 -0
@@ -120,6 +120,20 @@ xfs_rtgroup_get(return to_rtg (xfs_group_get (mp, rgno,XG_TYPE_RTG )); } +static inline int +xfs_rtgroup_get_passive_refcount (struct xfs_rtgroup *rtg) +{ +ASSERT (rtg); +return xfs_group_get_passive_refcount (rtg_group (rtg)); +} + +static inline int +xfs_rtgroup_get_active_refcount (struct xfs_rtgroup *rtg) +{ +ASSERT (rtg); +return xfs_group_get_active_refcount (rtg_group (rtg)); +} +static inline struct xfs_rtgroup *xfs_rtgroup_hold (struct xfs_rtgroup *rtg)
@@ -277,6 +291,21 @@ xfs_daddr_to_rtb(return bno; } +static inline bool +xfs_rtgroup_is_active (struct xfs_rtgroup *rtg) +{ +ASSERT (rtg); +return xfs_rtgroup_get_active_refcount (rtg) >0 ; +} + +void xfs_rtgroup_activate (struct xfs_rtgroup *rtg); +int xfs_rtgroup_deactivate (struct xfs_rtgroup *rtg); +bool xfs_rtgroup_is_empty (struct xfs_rtgroup *rtg); + +#define for_each_rgno_range_reverse (agno, old_rgcount, new_rgcount) \ + for ((agno) = ((old_rgcount) - 1); (typeof(old_rgcount))(agno) >= \ + ((typeof(old_rgcount))(new_rgcount) - 1); (agno)--) +#ifdef CONFIG_XFS_RT int xfs_rtgroup_alloc (struct xfs_mount *mp,xfs_rgnumber_t rgno,xfs_rgnumber_t rgcount,xfs_rtbxlen_t rextents);
fs/xfs/xfs_buf_item_recover.c +15 -10
@@ -750,11 +750,8 @@ xlog_recover_do_primary_sb_buffer(xfs_alert (mp,"Shrinking AG count in log recovery not supported" );return -EFSCORRUPTED ; } -if (mp->m_sb.sb_rgcount < orig_rgcount) { -xfs_warn (mp, -"Shrinking rtgroup count in log recovery not supported" ); -return -EFSCORRUPTED ; - } +if (mp->m_sb.sb_rgcount < orig_rgcount) +xfs_warn_experimental (mp,XFS_EXPERIMENTAL_SHRINK );/* * If the last AG was grown or shrunk, we also need to update the
@@ -789,11 +786,19 @@ xlog_recover_do_primary_sb_buffer( } mp->m_alloc_set_aside =xfs_alloc_set_aside (mp); - error =xfs_initialize_rtgroups (mp, orig_rgcount, mp->m_sb.sb_rgcount, - mp->m_sb.sb_rextents); -if (error) { -xfs_warn (mp,"Failed recovery rtgroup init: %d" , error); -return error; +if (orig_rgcount > mp->m_sb.sb_rgcount) { +/* +* Remove the old rtgroups that were removed previously by a +* growfs. +*/ +xfs_free_rtgroups (mp, mp->m_sb.sb_rgcount, orig_rgcount); + }else { + error =xfs_initialize_rtgroups (mp, orig_rgcount, + mp->m_sb.sb_rgcount, mp->m_sb.sb_rextents); +if (error) { +xfs_warn (mp,"Failed recovery rtgroup init: %d" , error); +return error; + } }return 0 ; }
fs/xfs/xfs_extent_busy.c +30 -0
@@ -676,6 +676,36 @@ xfs_extent_busy_wait_all(xfs_extent_busy_wait_group (rtg_group (rtg)); } +/* +* Similar to xfs_extent_busy_wait_all() - It waits for all the busy extents to +* get resolved for the range of rtgroups provided. For now, this function is +* introduced to be used in online shrink process. Unlike +* xfs_extent_busy_wait_all(), this takes a passive reference, because this +* function is expected to be called for the rtgroups whose active reference has +* been reduced to 0 i.e, offline rtgroups. +* +* @mp - The xfs mount point +* @first_agno - The 0 based AG index of the range of rtgroups from which we +* will start. +* @end_agno - The 0 based AG index of the range of rtgroups from till which we +* will traverse. +*/ +void +xfs_extent_busy_wait_rtgroups ( +struct xfs_mount *mp, +xfs_rgnumber_t first_rgno, +xfs_rgnumber_t end_rgno) +{ +xfs_agnumber_t rgno; +struct xfs_rtgroup *rtg =NULL ; + +for_each_rgno_range_reverse (rgno, end_rgno +1 , first_rgno +1 ) { + rtg =xfs_rtgroup_get (mp, rgno); +xfs_extent_busy_wait_group (rtg_group (rtg)); +xfs_rtgroup_put (rtg); + } +} +/* * Callback for list_sort to sort busy extents by the group they reside in. */
fs/xfs/xfs_extent_busy.h +2 -0
@@ -57,6 +57,8 @@ bool xfs_extent_busy_trim(struct xfs_group *xg, xfs_extlen_t minlen, unsigned *busy_gen);int xfs_extent_busy_flush (struct xfs_trans *tp,struct xfs_group *xg,unsigned busy_gen,uint32_t alloc_flags); +void xfs_extent_busy_wait_rtgroups (struct xfs_mount *mp, +xfs_rgnumber_t first_rgno,xfs_rgnumber_t end_rgno);void xfs_extent_busy_wait_all (struct xfs_mount *mp);bool xfs_extent_busy_list_empty (struct xfs_group *xg,unsigned int *busy_gen);struct xfs_extent_busy_tree *xfs_extent_busy_alloc (void );
fs/xfs/xfs_inode.c +6 -2
@@ -1410,8 +1410,12 @@ xfs_inactive(if (xfs_is_readonly (mp) && !xlog_recovery_needed (mp->m_log)) goto out; -/* Metadata inodes require explicit resource cleanup. */ -if (xfs_is_internal_inode (ip)) +/* +* Metadata inodes require explicit resource cleanup. During shrink we +* are using this function to remove the already truncated metadata +* inodes for the bitmap file, summary file. +*/ +if (xfs_is_internal_inode (ip) && !xfs_is_shrinking (mp)) goto out;/* Try to clean out the cow blocks if there are any. */
fs/xfs/xfs_rtalloc.c +795 -4
855 lines not on this page · raw message
fs/xfs/xfs_trans.c +0 -1
@@ -431,7 +431,6 @@ xfs_trans_mod_sb( tp->t_rextslog_delta += delta;break ;case XFS_TRANS_SB_RGCOUNT : -ASSERT (delta >0 ); tp->t_rgcount_delta += delta;break ;default :
Signature · 1 line
2.43.5
PATCH 1/1 xfs_growfs: Allow shrink of realtime XFS
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
This patch allows shrinking of realtime XFS ioctl to be passed to the kernel.
--- growfs/xfs_growfs.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
growfs/xfs_growfs.c +5 -4
@@ -296,11 +296,12 @@ _("[EXPERIMENTAL] try to shrink unused space %lld, old size is %lld\n"), error =1 ; }if (!error && rsize < geo.rtblocks) { -fprintf (stderr,_ ( -"realtime size %lld too small, old size is %lld\n" ), +fprintf (stderr, +_ ("[EXPERIMENTAL] try to shrink realtime unused space %lld, old size is %lld\n" ), (long long )rsize, (long long )geo.rtblocks); - error =1 ; - }else if (!error && rsize == geo.rtblocks) { + } + +if (!error && rsize == geo.rtblocks) {if (rflag)fprintf (stderr,_ ("realtime size unchanged, skipping\n" ));
Signature · 1 line
2.34.1
cover Add multi rtgroup grow and shrink tests
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink tests
common/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-) create mode 100755 tests/xfs/333 create mode 100644 tests/xfs/333.out create mode 100755 tests/xfs/539 create mode 100644 tests/xfs/539.out create mode 100755 tests/xfs/611 create mode 100644 tests/xfs/611.out create mode 100755 tests/xfs/654 create mode 100644 tests/xfs/654.out create mode 100755 tests/xfs/655 create mode 100644 tests/xfs/655.out
Signature · 1 line
2.34.1
PATCH 2/7 xfs: Introduce helpers to count the number of bitmap and summary inodes
From: "Nirjhar Roy (IBM)" <[email protected]>
With -metadir=1, we need some helpers that will count the number of bitmap and summary inodes for realtime XFS. This will be used in the tests for upcoming patches.
--- common/xfs | 12 ++++++++++++ 1 file changed, 12 insertions(+)
common/xfs +12 -0
@@ -1375,6 +1375,18 @@ _require_no_xfs_always_cow() fi } +# Counts number of bitmap files present in the metadata directory +_scratch_xfs_get_rt_bitmap_count() +{ + _scratch_xfs_db -r -c "ls -m /rtgroups" | grep "\.bitmap" | wc -l +} + +# Counts number of summary files present in the metadata directory +_scratch_xfs_get_rt_summary_count() +{ + _scratch_xfs_db -r -c "ls -m /rtgroups" | grep "\.summary" | wc -l +} + # Get a metadata field # The first arg is the field name # The rest of the arguments are xfs_db commands to find the metadata.
Signature · 1 line
2.34.1
PATCH 3/7 xfs: Add realtime group grow tests
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
Add unit tests to validate growth of realtime groups. This includes last rtgroup's partial growth case as well complete rtgroup addition.
--- tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 2 files changed, 164 insertions(+) create mode 100755 tests/xfs/655 create mode 100644 tests/xfs/655.out
tests/xfs/655 new file +151 -0
@@ -0,0 +1,151 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Nirjhar Roy (IBM) <[email protected]>. All Rights Reserved. +# +# FS QA Test 655 +# +# XFS realtime fs growfs basic functionality test +# +# This test attempts to grow the realtime filesystem by various sizes and +# verifies if it works as expected. +# +. ./common/preamble +_begin_fstest auto quick growfs shrinkfs + +# Import common functions. +. ./common/filter +_require_realtime_xfs_grow + +AGCOUNT=16 +RGCOUNT="$AGCOUNT" +DEVSIZE=$((2 * 1024 * 1024 * 1024)) # 2G +FSSIZE=$(( DEVSIZE / 2 )) +_require_scratch_size $((DEVSIZE / 1024)) + +_fixed_by_kernel_commit a65fd8120766 "xfs: Fix xfs_grow_last_rtg()" +_fixed_by_kernel_commit xxxxxxxxxxxx "xfs: Fix in xfs_rtalloc_query_range()" + +reset() +{ + _scratch_unmount >> $seqres.full 2>&1 + _scratch_mkfs -bsize=4k -dsize="$FSSIZE" -dagcount="$AGCOUNT" -rsize="$FSSIZE" \ + -mmetadir=1 -rrgcount="$RGCOUNT" 2>&1 | \ + tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs >/dev/null + . $tmp.mkfs + tot_rtextents=$rtextents + orgcount=$rgcount + _scratch_mount >> $seqres.full +} + +# This functions converts a given number of rtextents to rg count. +_rtextents_to_rg() +{ + local extents="$1" + local rgdiv=$((extents/rgextents)) + local rgmod=$((extents%rgextents)) + local rgc + if [[ "$rgmod" == 0 ]]; then + rgc="$rgdiv" + else + rgc=$(( rgdiv + 1 )) + fi + echo "$rgc" +} + +test_grow() +{ + $XFS_GROWFS_PROG -R"$1" $SCRATCH_MNT &> $tmp.growfs + ret=$? + + _scratch_unmount + _check_scratch_fs + _scratch_mount + + cat $tmp.growfs >> $seqres.full + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | \ + _filter_mkfs 2>$tmp.xfsinfo >/dev/null + . $tmp.xfsinfo + [ $ret -eq 0 ] +} + +# run_test <number of rtextents to add> <test name> <optional: error message> +# If the test is expected to succeed, then there is no need to provide an error +# message. +run_test_grow() +{ + local delta_extents="$1" + local test_name="$2" + local error_message="$3" + + echo "$test_name" + reset + local tsize=$((tot_rtextents+delta_extents)) + local nrgcount=`_rtextents_to_rg "$tsize"` + echo "delta = $delta_extents new_rtextents = $tsize nrgcount = $nrgcount" \ + >> $seqres.full + if [[ "$error_message" == "" ]]; then + # no error message means the test is expected to succeed + test_grow "$tsize" || \ + echo "Grow $test_name failure" + # Verify the parameters + [[ "$rtextents" -ne "$tsize" ]] && \ + echo "rtextents not changed properly after growing \"$test_name\"" \ + && echo "expected $tsize got $rtextents" + [[ "$rgcount" -ne "$nrgcount" ]] && \ + echo "rgcount not changed properly after growing \"$test_name\"" \ + && echo "expected $nrgcount got $rgcount" + [[ "$(_scratch_xfs_get_rt_bitmap_count)" -ne "$nrgcount" ]] && \ + echo "bitmap inode count not changed properly after growing \"$test_name\"" \ + && echo "expected $nrgcount got $(_scratch_xfs_get_rt_bitmap_count)" + [[ "$(_scratch_xfs_get_rt_summary_count)" -ne "$nrgcount" ]] && \ + echo "summary inode count not changed properly after growing \"$test_name\"" \ + && echo "expected $nrgcount got $(_scratch_xfs_get_rt_summary_count)" + else + test_grow "$tsize" && \ + echo "Grow \"$test_name\" succeeded unexpectedly" + grep -q "$error_message" $tmp.growfs || \ + echo "Error message missing - growing \"$test_name\" (expected: $error_message)" + # Verify the parameters + [[ "$rtextents" -ne "$tot_rtextents" ]] && \ + echo "rtextents changed after grow failure \"$test_name\"" && \ + echo "expected $tot_rtextents got $rtextents" + [[ "$rgcount" -ne "$orgcount" ]] && \ + echo "rgcount changed after grow failure \"$test_name\"" && \ + echo "expected $orgcount got $rgcount" + [[ "$(_scratch_xfs_get_rt_bitmap_count)" -ne "$orgcount" ]] && \ + echo "bitmap inode count changed after grow failure \"$test_name\"" && \ + echo "expected $orgcount got $(_scratch_xfs_get_rt_bitmap_count)" + [[ "$(_scratch_xfs_get_rt_summary_count)" -ne "$orgcount" ]] && \ + echo "summary inode count changed after grow failure \"$test_name\"" && \ + echo "expected $orgcount got $(_scratch_xfs_get_rt_summary_count)" + fi +} + +reset + +# Tests expected to pass +# run_test_grow <number of rtextents to add> <test name> +echo "#Tests partial and multi-rg grow" +run_test_grow 1 "grow by 1 rtextent" +run_test_grow $(( rgextents / 2 )) "grow by 0.5 RG" +run_test_grow $(( rgextents)) "grow by 1 RG" +run_test_grow $(( rgextents * 6 )) "grow by 6 RG" +run_test_grow $(( rgextents * 3 + rgextents / 2)) "grow by 3.5 RG" + +run_test_grow $(( dbsize * 8 )) "grow by 1 rmblocks" +run_test_grow $(( dbsize * 8 + 1 )) "grow by 1 + 1 rmblocks" +run_test_grow $(( rgextents + dbsize * 8 )) "grow by 1rtgroup + 1 rmblocks" + +echo +reset + +echo "#Test out of bounds rg grow: Below tests are expected to fail" +run_test_grow "$(( 9999999999 + 1 ))" "grow to > dev_size" \ + "too large" + +echo "printing seqres" >> $seqres.full + +$XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full + +# success, all done +_exit 0
tests/xfs/655.out new file +13 -0
@@ -0,0 +1,13 @@ +QA output created by 655 +#Tests partial and multi-rg grow +grow by 1 rtextent +grow by 0.5 RG +grow by 1 RG +grow by 6 RG +grow by 3.5 RG +grow by 1 rmblocks +grow by 1 + 1 rmblocks +grow by 1rtgroup + 1 rmblocks + +#Test out of bounds rg grow: Below tests are expected to fail +grow to > dev_size
Signature · 1 line
2.34.1
PATCH 1/7 xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
These is required for the set of XFS realtime grow/shrink tests in the upcoming patches.
--- common/xfs | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-)
common/xfs +52 -1
@@ -1233,7 +1233,11 @@ _require_xfs_mkfs_validation() _require_scratch_xfs_shrink() { + local is_realtime="$1" _require_scratch + if [ "$is_realtime" == "1" ]; then + _require_realtime + fi _require_command "$XFS_GROWFS_PROG" xfs_growfs _try_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs >/dev/null
@@ -1241,7 +1245,12 @@ _require_scratch_xfs_shrink() _scratch_mount # here just to check if kernel supports, no need do more extra work local errmsg - errmsg=$($XFS_GROWFS_PROG -D$((dblocks-1)) "$SCRATCH_MNT" 2>&1) + if [ "$is_realtime" == "1" ]; then + errmsg=$($XFS_GROWFS_PROG -R$((rtblocks-1)) "$SCRATCH_MNT" 2>&1) + else + errmsg=$($XFS_GROWFS_PROG -D$((dblocks-1)) "$SCRATCH_MNT" 2>&1) + fi + if [ "$?" -ne 0 ]; then echo "$errmsg" | grep 'XFS_IOC_FSGROWFSDATA xfsctl failed: Invalid argument' > /dev/null && \ _notrun "kernel does not support shrinking"
@@ -1251,6 +1260,48 @@ _require_scratch_xfs_shrink() fi _scratch_unmount } +_require_realtime_xfs_shrink() +{ + _require_scratch_xfs_shrink "1" +} + +_require_scratch_xfs_grow() +{ + local is_realtime="$1" + _require_scratch + if [ "$is_realtime" == "1" ]; then + _require_realtime + fi + _require_command "$XFS_GROWFS_PROG" xfs_growfs + + _try_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs >/dev/null + . $tmp.mkfs + _scratch_mount + # here just to check if kernel supports, no need do more extra work + local errmsg + if [ "$is_realtime" == "1" ]; then + errmsg=$($XFS_GROWFS_PROG -R$((rtblocks+1)) "$SCRATCH_MNT" 2>&1) + else + errmsg=$($XFS_GROWFS_PROG -D$((dblocks+1)) "$SCRATCH_MNT" 2>&1) + fi + + if [ "$?" -ne 0 ]; then + echo "$errmsg" | grep 'XFS_IOC_FSGROWFSDATA xfsctl failed: Invalid argument' > /dev/null && \ + _notrun "kernel does not support shrinking" + # if the fssize is already = backing device size, in that case, + # the above xfs_growfs will fail - but I don't want to the + # pre-condition to fail. This test case i.e, the new fs size + # greater than the backing device size is a test case that is + # covered in the test, so I will let the mount to take place so that the + # new fssize > device size can be actually be validated in a test case. + fi + _scratch_unmount +} + +_require_realtime_xfs_grow() +{ + _require_scratch_xfs_shrink "1" +} # this test requires mkfs.xfs have case-insensitive naming support _require_xfs_mkfs_ciname()
Signature · 1 line
2.34.1
PATCH 4/7 xfs: Add multi rt group grow + shutdown + recovery tests
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
This test repeatedly grows the realtime, then shuts down the filesystem and checks that recovery works fine - this is similar to what is already being done in xfs/609 but this tests with realtime XFS.
--- tests/xfs/654 | 90 +++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/654.out | 5 +++ 2 files changed, 95 insertions(+) create mode 100755 tests/xfs/654 create mode 100644 tests/xfs/654.out
tests/xfs/654 new file +90 -0
@@ -0,0 +1,90 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Nirjhar Roy (IBM) <[email protected]>. All Rights Reserved. +# +# FS QA Test 654 +# +# Test XFS online growfs log recover with growing the realtime filesystem +# +. ./common/preamble +_begin_fstest auto growfs stress shutdown log recoveryloop shrinkfs + +# Import common functions. +. ./common/filter + +_stress_scratch() +{ + procs=4 + nops=999 + # -w ensures that the only ops are ones which cause write I/O + FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -w -p $procs \ + -n $nops` + _run_fsstress_bg $FSSTRESS_ARGS >> $seqres.full 2>&1 +} +FSSIZE=$(( 2 * 1024 * 1024 * 1024 )) # 2G +_require_realtime_xfs_grow +_require_scratch_size $((FSSIZE / 1024)) #KB +_require_command "$XFS_GROWFS_PROG" xfs_growfs + +_fixed_by_kernel_commit a65fd8120766 "xfs: Fix xfs_grow_last_rtg()" +_fixed_by_kernel_commit xxxxxxxxxxxx "xfs: Fix in xfs_rtalloc_query_range()" + +_scratch_mkfs_xfs | _filter_mkfs >$seqres.full 2>$tmp.mkfs +. $tmp.mkfs + +endsize=`expr 1024 \* 1048576` # stop after shrinking 512M size +[ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small" + +nrgs=8 +nags="$nrgs" +# We want to test with and without stress. +for stress in 0 1 +do + for metadir in 0 1 + do + size=`expr 512 \* 1048576` # 512 megabytes initially + sizeb=`expr $size / $dbsize` # in data blocks + _scratch_mkfs_xfs -bsize=4k -dsize=${size} -dagcount=${nags} -rsize=${size} \ + -mmetadir="$metadir" -rrgcount=${nrgs} >> $seqres.full 2>&1 || _fail "mkfs failed" + _scratch_mount + + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.xfsinfo >/dev/null + . $tmp.xfsinfo + + if [[ "$stress" -eq 1 ]]; then + echo -n "Testing with stress " + else + echo -n "Testing without stress " + fi + + echo -n "with -m metadir=$metadir " + # Grow the filesystem in random sized chunks while performing shutdown and + # recovery. The randomization is intended to create a mix of sub-rtg and + # multi-rtg shrinks. + + while [ $size -le $endsize ]; do + echo "*** growing a ${sizeb} block filesystem " >> \ + $seqres.full + if [[ "$stress" -eq 1 ]]; then + _stress_scratch + fi + + incsize=$((RANDOM % 40 * 1048576)) + size=`expr $size + $incsize` + sizeb=`expr $size / $dbsize` + $XFS_GROWFS_PROG -R ${sizeb} $SCRATCH_MNT >> $seqres.full 2>&1 + + sleep $((RANDOM % 3)) + _scratch_shutdown + if [[ "$stress" -eq 1 ]]; then + _kill_fsstress + fi + _scratch_cycle_mount + done > /dev/null 2>&1 + wait + _scratch_unmount + echo "succeeded" + done +done + +_exit 0
tests/xfs/654.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 654 +Testing without stress with -m metadir=0 succeeded +Testing without stress with -m metadir=1 succeeded +Testing with stress with -m metadir=0 succeeded +Testing with stress with -m metadir=1 succeeded
Signature · 1 line
2.34.1
PATCH 5/7 xfs: Add realtime group shrink tests
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
Add unit tests to validate shrinking of realtime groups. This includes last rtgroup's partial shrink case as well compelete rtgroup removals.
--- tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ 2 files changed, 209 insertions(+) create mode 100755 tests/xfs/539 create mode 100644 tests/xfs/539.out
tests/xfs/539 new file +190 -0
@@ -0,0 +1,190 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Nirjhar Roy (IBM) <[email protected]>. All Rights Reserved. +# +# FS QA Test 539 +# +# XFS realtime fs shrinkfs basic functionality test +# +# This test attempts to shrink the realtime filesystem by various sizes and +# verifies if it works as expected. +# +. ./common/preamble +_begin_fstest auto quick growfs shrinkfs + +# Import common functions. +. ./common/filter +_require_realtime_xfs_shrink + +AGCOUNT=16 +RGCOUNT="$AGCOUNT" +FSSIZE=$((2 * 1024 * 1024 * 1024)) # 2G +_require_scratch_size $((FSSIZE / 1024)) + +_fixed_by_kernel_commit a65fd8120766 "xfs: Fix xfs_grow_last_rtg()" +_fixed_by_kernel_commit xxxxxxxxxxxx "xfs: Fix in xfs_rtalloc_query_range()" + +reset() +{ + _scratch_unmount >> $seqres.full 2>&1 + _scratch_mkfs -bsize=4k -dsize="$FSSIZE" -dagcount="$AGCOUNT" -rsize="$FSSIZE" \ + -mmetadir=1 -rrgcount="$RGCOUNT" 2>&1 | \ + tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs >/dev/null + . $tmp.mkfs + tot_rtextents=$rtextents + orgcount=$rgcount + _scratch_mount >> $seqres.full +} + +# This functions converts a given number of rtextents to rg count. +_rtextents_to_rg() +{ + local extents="$1" + local rgdiv=$((extents/rgextents)) + local rgmod=$((extents%rgextents)) + local rgc + if [[ "$rgmod" == 0 ]]; then + rgc="$rgdiv" + else + rgc=$(( rgdiv + 1 )) + fi + echo "$rgc" +} + +test_shrink() +{ + $XFS_GROWFS_PROG -R"$1" $SCRATCH_MNT &> $tmp.growfs + ret=$? + + _scratch_unmount + _check_scratch_fs + _scratch_mount + + cat $tmp.growfs >> $seqres.full + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | \ + _filter_mkfs 2>$tmp.xfsinfo >/dev/null + . $tmp.xfsinfo + [ $ret -eq 0 ] +} + +# run_test <number of rtextents to remove> <test name> <optional: error message> +# If the test is expected to succeed, then there is no need to provide an error +# message. +run_test_shrink() +{ + local delta_extents="$1" + local test_name="$2" + local error_message="$3" + + echo "$test_name" + reset + local tsize=$((tot_rtextents-delta_extents)) + local nrgcount=`_rtextents_to_rg "$tsize"` + echo "delta = $delta_extents new_rtextents = $tsize nrgcount = $nrgcount" \ + >> $seqres.full + if [[ "$error_message" == "" ]]; then + # no error message means the test is expected to succeed + test_shrink "$tsize" || \ + echo "Shrink $test_name failure" + # Verify the parameters + [[ "$rtextents" -ne "$tsize" ]] && \ + echo "rtextents not changed properly after shrinking \"$test_name\"" \ + && echo "expected $tsize got $rtextents" + [[ "$rgcount" -ne "$nrgcount" ]] && \ + echo "rgcount not changed properly after shrinking \"$test_name\"" \ + && echo "expected $nrgcount got $rgcount" + [[ "$(_scratch_xfs_get_rt_bitmap_count)" -ne "$nrgcount" ]] && \ + echo "bitmap inode count not changed properly after shrinking \"$test_name\"" \ + && echo "expected $nrgcount got $(_scratch_xfs_get_rt_bitmap_count)" + [[ "$(_scratch_xfs_get_rt_summary_count)" -ne "$nrgcount" ]] && \ + echo "summary inode count not changed properly after shrinking \"$test_name\"" \ + && echo "expected $nrgcount got $(_scratch_xfs_get_rt_summary_count)" + else + test_shrink "$tsize" && \ + echo "Shrink \"$test_name\" succeeded unexpectedly" + grep -q "$error_message" $tmp.growfs || \ + echo "Error message missing - shrinking \"$test_name\" (expected: $error_message)" + # Verify the parameters + [[ "$rtextents" -ne "$tot_rtextents" ]] && \ + echo "rtextents changed after shrink failure \"$test_name\"" && \ + echo "expected $tot_rtextents got $rtextents" + [[ "$rgcount" -ne "$orgcount" ]] && \ + echo "rgcount changed after shrink failure \"$test_name\"" && \ + echo "expected $orgcount got $rgcount" + [[ "$(_scratch_xfs_get_rt_bitmap_count)" -ne "$orgcount" ]] && \ + echo "bitmap inode count changed after shrink failure \"$test_name\"" && \ + echo "expected $orgcount got $(_scratch_xfs_get_rt_bitmap_count)" + [[ "$(_scratch_xfs_get_rt_summary_count)" -ne "$orgcount" ]] && \ + echo "summary inode count changed after shrink failure \"$test_name\"" && \ + echo "expected $orgcount got $(_scratch_xfs_get_rt_summary_count)" + fi +} + +# This function, first shrinks the fs, then grows it. This is to make sure that +# the fs is able to grow properly after a shrink has taken place. +# Usage: run_test_shrink_grow <number of rtextents to remove> <test name> +run_test_shrink_grow() +{ + local delta_extents="$1" + local test_name="$2" + + echo "$test_name" + reset + local tsize=$((tot_rtextents-delta_extents)) + test_shrink "$tsize" || \ + echo "$test_name failure" + # Now grow the fs + test_shrink "$tot_rtextents" || \ + echo "$test_name failure" + # Verify the parameters + [[ "$rtextents" -ne "$tot_rtextents" ]] && \ + echo "rtextents not changed properly after \"$test_name\"" \ + && echo "expected $tot_rtextents got $rtextents" + [[ "$rgcount" -ne "$orgcount" ]] && \ + echo "rgcount not changed properly after \"$test_name\"" \ + && echo "expected $orgcount got $rgcount" +} + +reset + +# Tests expected to pass +# run_test_shrink <number of rtextents to remove> <test name> +echo "#Tests partial and multi-rg shrinking" +run_test_shrink 1 "shrink by 1 rtextent" +run_test_shrink $(( rgextents / 2 )) "shrink by 0.5 RG" +run_test_shrink $(( rgextents)) "shrink by 1 RG" +run_test_shrink $(( rgextents * 6 )) "shrink by 6 RG" +run_test_shrink $(( rgextents * 3 + rgextents / 2)) "shrink by 3.5 RG" + +run_test_shrink $(( dbsize * 8 )) "shrink by 1 rmblocks" +run_test_shrink $(( rgextents + dbsize * 8 )) "shrink by 1rtgroup + 1 rmblocks" + +echo + +reset +# run_test_shrink_grow <number of rtextents to remove> <test name> +echo "#Tests partial and multi-rg shink followed by growfs" +run_test_shrink_grow $(( rgextents / 2 )) "shrink by 0.5 RG -> grow by 0.5 RG" +run_test_shrink_grow $(( rgextents * 6 )) "shrink by 6 RG -> grow by 6 RG" +run_test_shrink_grow $(( rgextents * 3 + rgextents / 2 )) \ + "shrink by 3.5 RG -> grow by 3.5 RG" +run_test_shrink_grow $(( dbsize * 8 )) "shrink by 1 rmblocks -> grow by 1 rmblocks" +run_test_shrink_grow $(( rgextents + dbsize * 8 )) \ + "shrink by 1rtgroup + 1 rmblocks -> grow by 1rtgroup + 1 rmblocks" + +echo + +reset +echo "#Test out of bounds rg shrink: Below tests are expected to fail" +# Tests expected to fail +# run_test_shrink_grow <number of rtextents to remove> <test name> <error message> +run_test_shrink "$(( 99999999 ))" "shrink to > dev size" \ + "File too large" + +echo "printing seqres" >> $seqres.full +cat $tmp.growfs >> $seqres.full + +$XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full + +# success, all done +_exit 0
tests/xfs/539.out new file +19 -0
@@ -0,0 +1,19 @@ +QA output created by 539 +#Tests partial and multi-rg shrinking +shrink by 1 rtextent +shrink by 0.5 RG +shrink by 1 RG +shrink by 6 RG +shrink by 3.5 RG +shrink by 1 rmblocks +shrink by 1rtgroup + 1 rmblocks + +#Tests partial and multi-rg shink followed by growfs +shrink by 0.5 RG -> grow by 0.5 RG +shrink by 6 RG -> grow by 6 RG +shrink by 3.5 RG -> grow by 3.5 RG +shrink by 1 rmblocks -> grow by 1 rmblocks +shrink by 1rtgroup + 1 rmblocks -> grow by 1rtgroup + 1 rmblocks + +#Test out of bounds rg shrink: Below tests are expected to fail +shrink to > dev size
Signature · 1 line
2.34.1
PATCH 6/7 xfs: Add multi rt group shrink + shutdown + recovery tests
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
This test repeatedly shrinks, then shuts down the filesystem and checks that recovery works fine - this is similar to what is already being done in xfs/609 but this tests with filesystem shrink and in xfs/609 it is tested with filesystem growth.
--- tests/xfs/333 | 95 +++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/333.out | 5 +++ 2 files changed, 100 insertions(+) create mode 100755 tests/xfs/333 create mode 100644 tests/xfs/333.out
tests/xfs/333 new file +95 -0
@@ -0,0 +1,95 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Nirjhar Roy (IBM) <[email protected]>. All Rights Reserved. +# +# FS QA Test 333 +# +# Test XFS online growfs log recover with shrinking the realtime filesystem +# +. ./common/preamble +_begin_fstest auto growfs stress shutdown log recoveryloop shrinkfs + +# Import common functions. +. ./common/filter + +_stress_scratch() +{ + procs=4 + nops=999 + # -w ensures that the only ops are ones which cause write I/O + FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -w -p $procs \ + -n $nops` + _run_fsstress_bg $FSSTRESS_ARGS >> $seqres.full 2>&1 +} +FSSIZE=$(( 1024 * 1024 * 1024 )) # 1G +_require_realtime_xfs_shrink +_require_scratch_size $((FSSIZE / 1024)) #KB +_require_command "$XFS_GROWFS_PROG" xfs_growfs + +_fixed_by_kernel_commit a65fd8120766 "xfs: Fix xfs_grow_last_rtg()" +_fixed_by_kernel_commit xxxxxxxxxxxx "xfs: Fix in xfs_rtalloc_query_range()" + +_scratch_mkfs_xfs | _filter_mkfs >$seqres.full 2>$tmp.mkfs +. $tmp.mkfs + +endsize=`expr 128 \* 1048576` # stop after shrinking to this size +[ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small" + +nrgs=16 +nags="$nrgs" +# We want to test with and without stress. The reason is that, if stress +# fills up the filesystem such that all the RGs are non-empty, then all +# the shrink commands will fail and we won't be able to test recovery +# during shrink. So to cover all the types of scenarios, we are testing +# with and without stress. +for stress in 0 1 +do + for metadir in 0 1 + do + size=`expr 512 \* 1048576` # 512 megabytes initially + sizeb=`expr $size / $dbsize` # in data blocks + _scratch_mkfs_xfs -bsize=4k -dsize=${size} -dagcount=${nags} -rsize=${size} \ + -mmetadir="$metadir" -rrgcount=${nrgs} >> $seqres.full 2>&1 || _fail "mkfs failed" + _scratch_mount + + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.xfsinfo >/dev/null + . $tmp.xfsinfo + + if [[ "$stress" -eq 1 ]]; then + echo -n "Testing with Stress " + else + echo -n "Testing without Stress " + fi + + echo -n "-m metadir = $metadir " + # Shrink the filesystem in random sized chunks while performing shutdown and + # recovery. The randomization is intended to create a mix of sub-rtg and + # multi-rtg shrinks. + + while [ $size -ge $endsize ]; do + echo "*** shrinking a ${sizeb} block filesystem (shrink)" >> \ + $seqres.full + if [[ "$stress" -eq 1 ]]; then + _stress_scratch + fi + + decsize=$((RANDOM % 40 * 1048576)) + size=`expr $size - $decsize` + sizeb=`expr $size / $dbsize` + $XFS_GROWFS_PROG -R ${sizeb} $SCRATCH_MNT >> $seqres.full 2>&1 + + sleep $((RANDOM % 3)) + _scratch_shutdown + if [[ "$stress" -eq 1 ]]; then + _kill_fsstress + fi + _scratch_cycle_mount + done > /dev/null 2>&1 + wait + _scratch_unmount + echo "succeeded" + done +done + +_exit 0 +
tests/xfs/333.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 333 +Testing without Stress -m metadir = 0 succeeded +Testing without Stress -m metadir = 1 succeeded +Testing with Stress -m metadir = 0 succeeded +Testing with Stress -m metadir = 1 succeeded
Signature · 1 line
2.34.1
PATCH 7/7 xfs: Add parallel back to back grow/shrink tests
In reply to Nirjhar Roy (IBM)From: "Nirjhar Roy (IBM)" <[email protected]>
This test makes several parallel back to back shrink/grow requests and makes sure that at the end of all operations, the filesystem is in a consistent state with no corruptions and crashes.
--- tests/xfs/611 | 97 +++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/611.out | 5 +++ 2 files changed, 102 insertions(+) create mode 100755 tests/xfs/611 create mode 100644 tests/xfs/611.out
tests/xfs/611 new file +97 -0
@@ -0,0 +1,97 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Nirjhar Roy (IBM) <[email protected]>. All Rights Reserved. +# +# FS QA Test 611 +# +# It creates NUM_PROCS number of processes. Some will pass, some will fail +# with: +# a) operation in progress +# b) no space available +# c) size unchanged. +# All the back to back xfs_growfs requests are made without recreating the +# filesystem in between 2 consecutive requests. This test ensures that even +# after several grow/shrink requests, the filesystem is in a consistent state +# and there are no crashes or metadata corruptions. + +. ./common/preamble +_begin_fstest auto quick growfs shrinkfs + +. ./common/filter +_require_realtime_xfs_shrink + +AGCOUNT=16 +RGCOUNT="$AGCOUNT" +FSSIZE=$(( 2 * 1024 * 1024 * 1024 )) # 2G +NUM_LOOP=500 +NUM_PROCS=$(( 4 * LOAD_FACTOR )) +_require_scratch_size $(( FSSIZE / 1024 )) #KB + +_fixed_by_kernel_commit a65fd8120766 "xfs: Fix xfs_grow_last_rtg()" +_fixed_by_kernel_commit xxxxxxxxxxxx "xfs: Fix in xfs_rtalloc_query_range()" + +_stress_scratch() +{ + procs=4 + nops=9999 + # -w ensures that the only ops are ones which cause write I/O + FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -w -p $procs \ + -n $nops` + _run_fsstress_bg $FSSTRESS_ARGS >> $seqres.full 2>&1 +} + +setup() +{ + _scratch_unmount $SCRATCH_MNT >> $seqres.full 2>&1 + _scratch_mkfs -bsize=4k -dsize="$FSSIZE" -dagcount="$AGCOUNT" -rsize="$FSSIZE"\ + -mmetadir="$1" -rrgcount="$RGCOUNT" 2>&1 | \ + tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs >/dev/null + . $tmp.mkfs + _scratch_mount >> $seqres.full + RGSZ=$(( FSSIZE / (dbsize * RGCOUNT) )) +} + +run_loop() { + for ((i=0; i<"$NUM_LOOP"; i++)); do + numag=$(( RANDOM % 17 )) + nblocksag=$(( numag * RGSZ )) + numblksoff=$(( RANDOM % RGSZ )) + nblkstot=$(( nblocksag + numblksoff )) + $XFS_GROWFS_PROG -R "$nblkstot" $SCRATCH_MNT >> $seqres.full 2>&1 + done +} + +start_test() +{ + for ((j=0; j<"$NUM_PROCS"; j++)); do + run_loop "$j" & + done + if [[ "$1" -eq 1 ]]; then + _stress_scratch + fi +} + +# We want to test with and without stress. The reason is that, if stress +# fills up the filesystem such that all the rtgs are non-empty, then all +# the shrink commands will fail and we won't be able to test. So to cover all +# the types of scenarios, we are testing with and without stress. +for stress in 0 1 +do + for metadir in 1 0 + do + setup "$metadir" "$stress" + if [[ "$stress" -eq 1 ]]; then + echo -n "Testing with stress" + else + echo -n "Testing without stress" + fi + echo -n " with -m metadir=$metadir " + + start_test "$stress" + wait + _check_scratch_fs + echo "succeeded" + done +done +# success, all done +_exit 0
tests/xfs/611.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 611 +Testing without stress with -m metadir=1 succeeded +Testing without stress with -m metadir=0 succeeded +Testing with stress with -m metadir=1 succeeded +Testing with stress with -m metadir=0 succeeded
Signature · 1 line
2.34.1
Re: [RFC v1 1/4] xfs: Re-introduce xg_active_wq field in struct xfs_group
In reply to Nirjhar Roy (IBM)Quoted 4 lines · Nirjhar Roy (IBM)
The reason for this change is that the online shrink code will wait for all the active references to come down to zero before actually starting the shrink process (only if the number of rtextents that we are trying to remove is worth 1 or more rtgroups).
I'll look over the actual usage, but please try to avoid bloating structures for very rare uses like this. I think this would be a perfect candidate for wait_var_event / wake_up_var.
Re: [RFC v1 2/4] xfs: Introduce xfs_rtginodes_ensure_all()
In reply to Nirjhar Roy (IBM)fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1301
Quoted 2 lines
fs/xfs/xfs_rtalloc.c +2
-0+int +xfs_rtginodes_ensure_all (struct xfs_rtgroup *rtg)
Please use the usual XFS function definition style.
fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1303
Quoted 2 lines
fs/xfs/xfs_rtalloc.c +2
-0+{ +int i =0 ;
The for loop already does this.
fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1306
Quoted 3 lines
fs/xfs/xfs_rtalloc.c +3
-0+int error =0 ; + +ASSERT (rtg);
The assert isn't needed.
fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1313
Quoted 7 lines
fs/xfs/xfs_rtalloc.c +7
-0+ +for (i =0 ; i <XFS_RTGI_MAX ; i++) { + error =xfs_rtginode_ensure (rtg, i); +if (error) +break ; + } +return error;
Just return the error directly from the loop and 0 if you make it through, then there is no need for the error initialization at the top either.
Re: [RFC v1 4/4] xfs: Add support to shrink multiple empty realtime groups
In reply to Nirjhar Roy (IBM)Just q quick glance, no real review:
fs/xfs/libxfs/xfs_group.c xfs_group_get_by_fsb( · line 245
Quoted 12 lines
fs/xfs/libxfs/xfs_group.c +12
-0+int +xfs_group_get_active_refcount (struct xfs_group *xg) +{ +ASSERT (xg); +return atomic_read (&xg->xg_active_ref); +} + +int +xfs_group_get_passive_refcount (struct xfs_group *xg) +{ +ASSERT (xg); +return atomic_read (&xg->xg_ref);
Using "get" to read a refcount value is very confusing. Looking at the users I'm tempted to say just open code these and the ag/rtg wrappers.
fs/xfs/libxfs/xfs_rtgroup.c xfs_update_last_rtgroup_size( · line 188
Quoted 6 lines
fs/xfs/libxfs/xfs_rtgroup.c +5
-1- rtg =xfs_rtgroup_grab (mp, prev_rgcount -1 ); +if (prev_rgcount >= mp->m_sb.sb_rgcount) + rgno = mp->m_sb.sb_rgcount -1 ; +else + rgno = prev_rgcount -1 ; + rtg =xfs_rtgroup_grab (mp, rgno);
Throw in a comment that this is about grow/shrink?
Quoted 1 line
+void
Stick to the unique XFS style here and in other places, please.
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 774
Quoted 11 lines
fs/xfs/libxfs/xfs_rtgroup.c +11
-0+xfs_rtgroup_activate(struct xfs_rtgroup *rtg) +{ +ASSERT (!xfs_rtgroup_is_active (rtg)); +init_waitqueue_head (&rtg_group (rtg)->xg_active_wq); +atomic_set (&rtg_group (rtg)->xg_active_ref,1 ); +xfs_add_frextents (rtg_mount (rtg), +xfs_rtgroup_extents (rtg_mount (rtg),rtg_rgno (rtg))); +} + +int +xfs_rtgroup_deactivate (struct xfs_rtgroup *rtg)
It might also make sense to explain what activate/deactive means here.
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 778
Quoted 4 lines
fs/xfs/libxfs/xfs_rtgroup.c +4
-0+{ +ASSERT (rtg); + +int error =0 ;
No need for the assert, and code goes after declarations.
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 782
Quoted 4 lines
fs/xfs/libxfs/xfs_rtgroup.c +4
-0+xfs_rgnumber_t rgno =rtg_rgno (rtg); +struct xfs_mount *mp =rtg_mount (rtg); +xfs_rtxnum_t rtextents = +xfs_rtgroup_extents (mp, rgno);
This assignment fits onto a single line easily.
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 796
Quoted 13 lines
fs/xfs/libxfs/xfs_rtgroup.c +13
-0+ASSERT (xfs_rtgroup_is_active (rtg)); +ASSERT (rtg_rgno (rtg) < mp->m_sb.sb_rgcount); + +if (!xfs_rtgroup_is_empty (rtg)) +return -ENOTEMPTY ; +/* +* Manually reduce/reserve 1 realtime group worth of +* free realtime extents from the global counters. This is necessary +* in order to prevent a race where, some rtgs have been temporarily +* offlined but the delayed allocator has already promised some bytes +* and later the real extent/block allocation is failing due to +* the rtgs(s) being offline. +* If the overall shrink fails, we will restore the values.
Formatting: use up all 80 characters.
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 834
Quoted 3 lines
fs/xfs/libxfs/xfs_rtgroup.c +3
-0+xfs_rgnumber_t rgno =rtg_rgno (rtg); + +struct xfs_rtalloc_args args = {
Weird empty line between the declarations.
fs/xfs/libxfs/xfs_rtgroup.h xfs_daddr_to_rtb( · line 307
Quoted 5 lines
fs/xfs/libxfs/xfs_rtgroup.h +5
-0+bool xfs_rtgroup_is_empty (struct xfs_rtgroup *rtg); + +#define for_each_rgno_range_reverse (agno, old_rgcount, new_rgcount) \ + for ((agno) = ((old_rgcount) - 1); (typeof(old_rgcount))(agno) >= \ + ((typeof(old_rgcount))(new_rgcount) - 1); (agno)--)
I don't think this is helpful vs just open coding the loop. The mix of ag and rg naming is also a bit odd.
Review
Quoted 2 lines
(quoted diff) +2
-0+ for_each_rgno_range_reverse(rgno, old_rgcount, new_rgcount + 1) { + rtg = xfs_rtgroup_get(mp, rgno);
Given that we have this pattern a bit, maybe add a reverse version of xfs_group_next_range to encapsulate it?
Highlevel note: this seems to only cover classic bitmap based RTGs, and not zoned ones. You might want to look into the latter because they're actually much simpler, and with the zoned GC code we actually have a super nice way to move data out of the RTG. I'd be happy to supple you a code sniplet for the latter.
Re: [PATCH v1 3/7] xfs: Add realtime group grow tests
In reply to Nirjhar Roy (IBM)Quoted 4 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
Add unit tests to validate growth of realtime groups. This includes last rtgroup's partial growth case as well complete rtgroup addition.
Please tests these also with zoned rt devices. They are a bit special in that you can only grow (and in the future shrink) entire RTGs, and don't have the bitmap/summary. For some cases that might mean just not running them (if they grow inside a RTG), and for others they might need very minor tweaks.
Carlos Maiolino
maintainerRe: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Nirjhar Roy (IBM)Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
Quoted 38 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-) create mode 100755 tests/xfs/333 create mode 100644 tests/xfs/333.out create mode 100755 tests/xfs/539 create mode 100644 tests/xfs/539.out create mode 100755 tests/xfs/611 create mode 100644 tests/xfs/611.out create mode 100755 tests/xfs/654 create mode 100644 tests/xfs/654.out create mode 100755 tests/xfs/655 create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
--NR
Quoted 38 lines
Quoted 38 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1
Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Carlos Maiolino
maintainerRe: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
Quoted 12 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.
Quoted 7 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Quoted 46 lines
--NR
Quoted 38 lines
Quoted 38 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Carlos MaiolinoQuoted 11 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
b4 has --no-parent argument for am/shazam to break the thread, pretty handy in such cases :)
--no-parent Break thread at the msgid specified and ignore any parent messages
Signature · 1 line
- Andrey
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Carlos MaiolinoQuoted 28 lines · Carlos Maiolino
Quoted 12 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 7 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
Quoted 8 lines
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Pull requests, perhaps?
Or is the problem here that you're using b4/korgalore/etc to download patchmails so that you can read them outside of a MUA?
((Again, I'll express a wish that people push their branches to git.kernel.org and send a link in the cover letter; that's much easier for me to pull and examine than reading emails or prying individual maybe-MTA-corrupted emails out of mutt into applyable form...))
--D
Quoted 48 lines
Quoted 47 lines
--NR
Quoted 38 lines
Quoted 38 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Carlos Maiolino
maintainerRe: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
Quoted 34 lines · Darrick J. Wong
Quoted 33 lines · Carlos Maiolino
Quoted 12 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 7 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
Quoted 8 lines
Quoted 3 lines
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Pull requests, perhaps?
Or is the problem here that you're using b4/korgalore/etc to download patchmails so that you can read them outside of a MUA?
Yup, b4. Andrey mentioned there is a way to break threads, but that bit me once pulling weird stuff.
Quoted 57 lines
((Again, I'll express a wish that people push their branches to git.kernel.org and send a link in the cover letter; that's much easier for me to pull and examine than reading emails or prying individual maybe-MTA-corrupted emails out of mutt into applyable form...))
--D
Quoted 48 lines
Quoted 47 lines
--NR
Quoted 38 lines
Quoted 38 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Carlos MaiolinoQuoted 27 lines · Carlos Maiolino
Quoted 11 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 7 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
Quoted 8 lines
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Okay sure. Actually I have seen some patchbombs follow a similar style i.e, have a top level root email/cover, and under that have multiple patchsets with each patchset having multiple patches/commits - so overall a 3 level nesting. Since this work has patches in 3 different projects, I thought of having all of them under one roof. If that is inconvenient for you, I will send individual patchsets separately from the next revision.
--NR
Quoted 46 lines
Quoted 46 lines
--NR
Quoted 37 lines
Quoted 37 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Darrick J. WongQuoted 39 lines · Darrick J. Wong
Quoted 27 lines · Carlos Maiolino
Quoted 11 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 7 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
Quoted 8 lines
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Pull requests, perhaps?
Quoted 7 lines
Or is the problem here that you're using b4/korgalore/etc to download patchmails so that you can read them outside of a MUA?
((Again, I'll express a wish that people push their branches to git.kernel.org and send a link in the cover letter; that's much easier for me to pull and examine than reading emails or prying individual maybe-MTA-corrupted emails out of mutt into applyable form...))
Okay, I will keep this in mind. I was under the impression that only maintainers can have their private branches in git.kernel.org. I will check on how to create a branch in git.kernel.org. For now, is it okay to just send the branch link as a separate email reply and you can take a look - next revision onwards I will have the link in the cover letter itself?
--NR
Quoted 47 lines
--D
Quoted 44 lines
Quoted 44 lines
--NR
Quoted 37 lines
Quoted 37 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Carlos MaiolinoQuoted 11 lines · Carlos Maiolino
Quoted 7 lines
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
I'd like to hear more of this or see some examples. The workflow of replying as follow-ups is adopted in many other projects (e.g. git) and they haven't reported any difficulties. The only time I'm aware of complications is when follow-up series are not properly versioned.
Regards,
Signature · 1 line
KR
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Nirjhar Roy (IBM)Quoted 43 lines · Nirjhar Roy (IBM)
Quoted 42 lines · Darrick J. Wong
Quoted 27 lines · Carlos Maiolino
Quoted 11 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 7 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
Quoted 8 lines
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Pull requests, perhaps?
Or is the problem here that you're using b4/korgalore/etc to download patchmails so that you can read them outside of a MUA?
Quoted 7 lines
Quoted 4 lines
((Again, I'll express a wish that people push their branches to git.kernel.org and send a link in the cover letter; that's much easier for me to pull and examine than reading emails or prying individual maybe-MTA-corrupted emails out of mutt into applyable form...))
Okay, I will keep this in mind. I was under the impression that only maintainers can have their private branches in git.kernel.org. I will
Anyone with a kernel.org account can push git repos.
github kinda works too, though I am a Bad Microsoft Citizen(tm) so they arbitrarily claim on first access that I'm over a rate limit that they do not specify. Granted that makes them a Bad Provider(tm) but I doubt they hear me above all the AI noise.
Quoted 4 lines
check on how to create a branch in git.kernel.org. For now, is it okay to just send the branch link as a separate email reply and you can take a look - next revision onwards I will have the link in the cover letter itself?
Sounds fine to me.
--D
Quoted 50 lines
--NR
Quoted 47 lines
--D
Quoted 44 lines
Quoted 44 lines
--NR
Quoted 37 lines
Quoted 37 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Nirjhar Roy (IBM)Quoted 37 lines · Nirjhar Roy (IBM)
Quoted 27 lines · Carlos Maiolino
Quoted 11 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 7 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
Quoted 8 lines
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Quoted 7 lines
Okay sure. Actually I have seen some patchbombs follow a similar style i.e, have a top level root email/cover, and under that have multiple patchsets with each patchset having multiple patches/commits - so overall a 3 level nesting. Since this work has patches in 3 different projects, I thought of having all of them under one roof. If that is inconvenient for you, I will send individual patchsets separately from the next revision.
Yeah, I do this too (combined kernel/xfsprogs/fstests patchsets under one giant cover letter) because a manager wanted a one-keystroke way to mark the entire megathread as read.
--D
Quoted 49 lines
--NR
Quoted 46 lines
Quoted 46 lines
--NR
Quoted 37 lines
Quoted 37 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Darrick J. WongQuoted 41 lines · Darrick J. Wong
Quoted 40 lines · Nirjhar Roy (IBM)
Quoted 39 lines · Darrick J. Wong
Quoted 25 lines · Carlos Maiolino
Quoted 11 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 6 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
Quoted 8 lines
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Pull requests, perhaps?
Or is the problem here that you're using b4/korgalore/etc to download patchmails so that you can read them outside of a MUA?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
((Again, I'll express a wish that people push their branches to git.kernel.org and send a link in the cover letter; that's much easier for me to pull and examine than reading emails or prying individual maybe-MTA-corrupted emails out of mutt into applyable form...))
Okay, I will keep this in mind. I was under the impression that only maintainers can have their private branches in git.kernel.org. I will
Anyone with a kernel.org account can push git repos.
https://korg.docs.kernel.org/accounts.html
It says the following:
Getting a kernel.org account
Kernel.org accounts are reserved for Linux kernel maintainers or high-profile developers. If you do not fall under one of these two categories, it’s unlikely that an account will be issued to you.
In that case, should I use GitHub?
Quoted 5 lines
github kinda works too, though I am a Bad Microsoft Citizen(tm) so they arbitrarily claim on first access that I'm over a rate limit that they do not specify. Granted that makes them a Bad Provider(tm) but I doubt they hear me above all the AI noise.
Okay.
Quoted 6 lines
Quoted 5 lines
check on how to create a branch in git.kernel.org. For now, is it okay to just send the branch link as a separate email reply and you can take a look - next revision onwards I will have the link in the cover letter itself?
Sounds fine to me.
Thank you.
--NR
Quoted 51 lines
--D
Quoted 48 lines
--NR
Quoted 46 lines
--D
Quoted 44 lines
Quoted 44 lines
--NR
Quoted 37 lines
Quoted 37 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [RFC v1 0/4] xfs: Add support to shrink multiple empty rtgroups
In reply to Nirjhar Roy (IBM)Quoted 37 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This work is based on a previous RFC[1] by Gao Xiang and various ideas proposed by Dave Chinner in the RFC[1].
The patch begins with the re-introduction of some of the data structures that were removed, some code refactoring and finally the patch that implements the multi rtgroup shrink design. We can remove only those rtgroups which are empty.
For non-empty rtgroups the rtextents can migrated to other rtgroups (assuming the required amount of space is available).
I am working on the patch series [2](by Darrick and
Dave) for completion of the data block migration from the end of the filesystem for emptying an realtime group. That will be a future work after this.
The final patch has all the details including the definition of the terminologies and the overall design.The design is quite similar to the design of data AG removal implemented in the patch series[3] that I have posted earlier.
The reason for keeping [3] on hold and posting this patch series is that (based on the discussion in [5], [6]), realtime devices won't have any metadata/inodes and migrating data from the end of realtime devices will be simpler. On the other hand there are challenges in moving metadata from regular AGs especially inodes.Please note that I have added RBs from Darrick in patch 1 which was given in [4].
Instructions to test this patch:
$ Apply the patch for xfsprogs sent with this series and install it. $ mkfs.xfs -f -m metadir=1 -r rtdev=/dev/loop2,extsize=4096,rgcount=4,size=1G \ -d size=1G /dev/loop1 $ mount -o rtdev=/dev/loop2 /dev/loop1 /mnt/scratch $ # shrink by 1.5 rtgroups $ xfs_growfs -R $(( 65536 * 2 + 32768 )) /mnt1/scratch
Quoted 8 lines
I have also sent the tests.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-xfs/173568777852.2709794.6356870909327619205.stgit@frogsfrogsfrogs/
[3] https://lore.kernel.org/linux-xfs/[email protected]/
[4] https://lore.kernel.org/all/20250729202632.GF2672049@frogsfrogsfrogs/
[5] https://lore.kernel.org/linux-xfs/[email protected]/
[6] https://lore.kernel.org/linux-xfs/[email protected]/
Hi Darrick,
Please find the github branches for the changes in the patchsets:
kernel changes - https://github.com/Nirjhar-Roy-0211/linux/commits/xfs-shrink-rt-v1/
userpace changes - https://github.com/Nirjhar-Roy-0211/xfsprogs-dev/commits/xfs-shrink-rt-v1/
fstests - https://github.com/Nirjhar-Roy-0211/xfstests-dev/commits/xfs-shrink-rt-v1/
--NR
Quoted 23 lines
Nirjhar Roy (IBM) (4):
xfs: Re-introducexg_active_wqfield in structxfs_group
xfs: Introducexfs_rtginodes_ensure_all()
xfs: Add a new state for shrinking
xfs: Add support to shrink multiple empty realtime groupsfs/xfs/libxfs/xfs_group.c | 18 +- fs/xfs/libxfs/xfs_group.h | 4 + fs/xfs/libxfs/xfs_rtgroup.c | 105 ++++- fs/xfs/libxfs/xfs_rtgroup.h | 31 ++ fs/xfs/xfs_buf_item_recover.c | 25 +- fs/xfs/xfs_extent_busy.c | 30 ++ fs/xfs/xfs_extent_busy.h | 2 + fs/xfs/xfs_inode.c | 8 +- fs/xfs/xfs_mount.h | 3 + fs/xfs/xfs_rtalloc.c | 824 +++++++++++++++++++++++++++++++++- fs/xfs/xfs_trans.c | 1 - 11 files changed, 1023 insertions(+), 28 deletions(-)--
2.43.5
Carlos Maiolino
maintainerRe: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Konstantin RyabitsevQuoted 9 lines · Konstantin Ryabitsev
Quoted 8 lines · Carlos Maiolino
Quoted 7 lines
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.
Quoted 8 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
I'd like to hear more of this or see some examples. The workflow of replying as follow-ups is adopted in many other projects (e.g. git) and they haven't reported any difficulties. The only time I'm aware of complications is when follow-up series are not properly versioned.
That might be indeed the case then. I just don't have any specific case to point you now, as this is not something that we see too often on linux-xfs. But a few times I tried to pull series that were in-reply-to their old versions, I ended up with the original version, or, IIRC, once, I kind of ended up with a mixed of patches from both versions.
When there are two versions of the same series in the list, b4 can easily find out the latest one, but seemed to have trouble (at least in the past) when they were in-reply-to.
Carlos Maiolino
maintainerRe: [PATCH v1 0/7] Add multi rtgroup grow and shrink tests
In reply to Nirjhar Roy (IBM)Quoted 55 lines · Nirjhar Roy (IBM)
Quoted 41 lines · Darrick J. Wong
Quoted 40 lines · Nirjhar Roy (IBM)
Quoted 39 lines · Darrick J. Wong
Quoted 25 lines · Carlos Maiolino
Quoted 11 lines · Nirjhar Roy (IBM)
Quoted 10 lines · Carlos Maiolino
Quoted 9 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This series adds several tests to validate the XFS realtime fs growth and shrink functionality.
It begins with the introduction of some preconditions and helper functions, then some tests that validate realtime group growth, followed by realtime group shrink/removal tests and ends with a test that validates both growth and shrink functionality together. Individual patches have the details.Quoted 6 lines
Quoted 3 lines
Please don't send new versions in reply to the old one, it just make hard to pull patches from the list. b4 usually doesn't handle it gracefully.
This entire series is new i.e, the kernel changes, fstests and the xfsprogs changes. Can you please explain as to what do you mean by the old version? Which old are version are you referring to?
Sure, I said 'old version' but the same applies to sending them in reply to other series/patches.
This series was sent:
In-Reply-To: <[email protected]>
Quoted 8 lines
which is:
Subject: xfs: Add support for multi rtgroup shrink+removal
In better wording, please don't nest series under other series/patches, or things like that. It works in some point cases, but in general it just makes my life difficult to pull them from the list.
Pull requests, perhaps?
Or is the problem here that you're using b4/korgalore/etc to download patchmails so that you can read them outside of a MUA?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
((Again, I'll express a wish that people push their branches to git.kernel.org and send a link in the cover letter; that's much easier for me to pull and examine than reading emails or prying individual maybe-MTA-corrupted emails out of mutt into applyable form...))
Okay, I will keep this in mind. I was under the impression that only maintainers can have their private branches in git.kernel.org. I will
Anyone with a kernel.org account can push git repos.
https://korg.docs.kernel.org/accounts.html
It says the following:
Quoted 7 lines
Getting a kernel.org account
Kernel.org accounts are reserved for Linux kernel maintainers or high-profile developers. If you do not fall under one of these two categories, it’s unlikely that an account will be issued to you.In that case, should I use GitHub?
No, I'm not taking pull requests from github. Please keep sending patches to the list.
Quoted 75 lines
Quoted 6 lines
github kinda works too, though I am a Bad Microsoft Citizen(tm) so they arbitrarily claim on first access that I'm over a rate limit that they do not specify. Granted that makes them a Bad Provider(tm) but I doubt they hear me above all the AI noise.
Okay.
Quoted 6 lines
Quoted 5 lines
check on how to create a branch in git.kernel.org. For now, is it okay to just send the branch link as a separate email reply and you can take a look - next revision onwards I will have the link in the cover letter itself?
Sounds fine to me.
Thank you.
--NR
Quoted 51 lines
--D
Quoted 48 lines
--NR
Quoted 46 lines
--D
Quoted 44 lines
Quoted 44 lines
--NR
Quoted 37 lines
Quoted 37 lines
Nirjhar Roy (IBM) (7):
xfs: Introduce _require_realtime_xfs_{shrink,grow} pre-condition
xfs: Introduce helpers to count the number of bitmap and summary
inodes
xfs: Add realtime group grow tests
xfs: Add multi rt group grow + shutdown + recovery tests
xfs: Add realtime group shrink tests
xfs: Add multi rt group shrink + shutdown + recovery tests
xfs: Add parallel back to back grow/shrink testscommon/xfs | 65 +++++++++++++++- tests/xfs/333 | 95 +++++++++++++++++++++++ tests/xfs/333.out | 5 ++ tests/xfs/539 | 190 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/539.out | 19 +++++ tests/xfs/611 | 97 +++++++++++++++++++++++ tests/xfs/611.out | 5 ++ tests/xfs/654 | 90 ++++++++++++++++++++++ tests/xfs/654.out | 5 ++ tests/xfs/655 | 151 ++++++++++++++++++++++++++++++++++++ tests/xfs/655.out | 13 ++++ 11 files changed, 734 insertions(+), 1 deletion(-)create mode 100755 tests/xfs/333
create mode 100644 tests/xfs/333.out
create mode 100755 tests/xfs/539
create mode 100644 tests/xfs/539.out
create mode 100755 tests/xfs/611
create mode 100644 tests/xfs/611.out
create mode 100755 tests/xfs/654
create mode 100644 tests/xfs/654.out
create mode 100755 tests/xfs/655
create mode 100644 tests/xfs/655.outSignature · 1 line
2.34.1Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, BangaloreSignature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [RFC v1 4/4] xfs: Add support to shrink multiple empty realtime groups
In reply to Christoph HellwigQuoted 18 lines · Christoph Hellwig
Just q quick glance, no real review:
fs/xfs/libxfs/xfs_group.c xfs_group_get_by_fsb( · line 245
Quoted 12 lines
fs/xfs/libxfs/xfs_group.c +12
-0+int +xfs_group_get_active_refcount (struct xfs_group *xg) +{ +ASSERT (xg); +return atomic_read (&xg->xg_active_ref); +} + +int +xfs_group_get_passive_refcount (struct xfs_group *xg) +{ +ASSERT (xg); +return atomic_read (&xg->xg_ref);Using "get" to read a refcount value is very confusing. Looking at the users I'm tempted to say just open code these and the ag/rtg wrappers.
Noted.
Quoted 9 lines
fs/xfs/libxfs/xfs_rtgroup.c xfs_update_last_rtgroup_size( · line 188
Quoted 7 lines
fs/xfs/libxfs/xfs_rtgroup.c +5
-1- rtg =xfs_rtgroup_grab (mp, prev_rgcount -1 ); +if (prev_rgcount >= mp->m_sb.sb_rgcount) + rgno = mp->m_sb.sb_rgcount -1 ; +else + rgno = prev_rgcount -1 ; + rtg =xfs_rtgroup_grab (mp, rgno);Throw in a comment that this is about grow/shrink?
Noted.
Quoted 4 lines
Quoted 2 lines
+void
Stick to the unique XFS style here and in other places, please.
Sure.
Quoted 14 lines
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 774
Quoted 12 lines
fs/xfs/libxfs/xfs_rtgroup.c +11
-0+xfs_rtgroup_activate(struct xfs_rtgroup *rtg) +{ +ASSERT (!xfs_rtgroup_is_active (rtg)); +init_waitqueue_head (&rtg_group (rtg)->xg_active_wq); +atomic_set (&rtg_group (rtg)->xg_active_ref,1 ); +xfs_add_frextents (rtg_mount (rtg), +xfs_rtgroup_extents (rtg_mount (rtg),rtg_rgno (rtg))); +} + +int +xfs_rtgroup_deactivate (struct xfs_rtgroup *rtg)It might also make sense to explain what activate/deactive means here.
Okay, will update in the upcoming revisions.
Quoted 7 lines
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 778
Quoted 5 lines
fs/xfs/libxfs/xfs_rtgroup.c +4
-0+{ +ASSERT (rtg); + +int error =0 ;No need for the assert, and code goes after declarations.
Noted.
Quoted 7 lines
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 782
Quoted 5 lines
fs/xfs/libxfs/xfs_rtgroup.c +4
-0+xfs_rgnumber_t rgno =rtg_rgno (rtg); +struct xfs_mount *mp =rtg_mount (rtg); +xfs_rtxnum_t rtextents = +xfs_rtgroup_extents (mp, rgno);This assignment fits onto a single line easily.
Okay - will fix this in the next revision.
Quoted 16 lines
Quoted 4 lines
fs/xfs/libxfs/xfs_rtgroup.c +3
-0+ASSERT (xfs_rtgroup_is_active (rtg)); +ASSERT (rtg_rgno (rtg) < mp->m_sb.sb_rgcount); +fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 796
Quoted 10 lines
fs/xfs/libxfs/xfs_rtgroup.c +10
-0+if (!xfs_rtgroup_is_empty (rtg)) +return -ENOTEMPTY ; +/* +* Manually reduce/reserve 1 realtime group worth of +* free realtime extents from the global counters. This is necessary +* in order to prevent a race where, some rtgs have been temporarily +* offlined but the delayed allocator has already promised some bytes +* and later the real extent/block allocation is failing due to +* the rtgs(s) being offline. +* If the overall shrink fails, we will restore the values. Formatting: use up all 80 characters.
Okay.
Quoted 6 lines
fs/xfs/libxfs/xfs_rtgroup.c xfs_log_rtsb( · line 834
Quoted 4 lines
fs/xfs/libxfs/xfs_rtgroup.c +3
-0+xfs_rgnumber_t rgno =rtg_rgno (rtg); + +struct xfs_rtalloc_args args = {Weird empty line between the declarations.
Will fix this.
Quoted 10 lines
fs/xfs/libxfs/xfs_rtgroup.h xfs_daddr_to_rtb( · line 307
Quoted 6 lines
fs/xfs/libxfs/xfs_rtgroup.h +5
-0+bool xfs_rtgroup_is_empty (struct xfs_rtgroup *rtg); + +#define for_each_rgno_range_reverse (agno, old_rgcount, new_rgcount) \ + for ((agno) = ((old_rgcount) - 1); (typeof(old_rgcount))(agno) >= \ + ((typeof(old_rgcount))(new_rgcount) - 1); (agno)--)I don't think this is helpful vs just open coding the loop. The mix of ag and rg naming is also a bit odd.
Quoted 5 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ for_each_rgno_range_reverse(rgno, old_rgcount, new_rgcount + 1) { + rtg = xfs_rtgroup_get(mp, rgno);Given that we have this pattern a bit, maybe add a reverse version of
xfs_group_next_rangeto encapsulate it?
Okay - I will try to come up with something. Thank you for the suggestion.
Quoted 4 lines
Highlevel note: this seems to only cover classic bitmap based RTGs, and not zoned ones. You might want to look into the latter because
Okay. So are you suggesting to add the shrink support only for the zoned devices or extend this patch set to cover shrink for the zoned devices too?
Quoted 3 lines
they're actually much simpler, and with the zoned GC code we actually have a super nice way to move data out of the RTG. I'd be happy to supple you a code sniplet for the latter.
Sure, that would be super useful. Also, since I don't have much experience in zoned devices, can you please point to some relevant resources and recent zoned device related patchsets that can help me get started with?
--NR
Re: [RFC v1 2/4] xfs: Introduce xfs_rtginodes_ensure_all()
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1301
Quoted 2 lines
fs/xfs/xfs_rtalloc.c +2
-0+int +xfs_rtginodes_ensure_all (struct xfs_rtgroup *rtg)Please use the usual XFS function definition style.
Will fix this.
Quoted 12 lines
fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1303
Quoted 3 lines
fs/xfs/xfs_rtalloc.c +2
-0+{ +int i =0 ;The for loop already does this.
fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1306
Quoted 3 lines
fs/xfs/xfs_rtalloc.c +3
-0+int error =0 ; + +ASSERT (rtg);The assert isn't needed.
Quoted 11 lines
fs/xfs/xfs_rtalloc.c xfs_growfs_check_rtgeom( · line 1313
Quoted 7 lines
fs/xfs/xfs_rtalloc.c +7
-0+ +for (i =0 ; i <XFS_RTGI_MAX ; i++) { + error =xfs_rtginode_ensure (rtg, i); +if (error) +break ; + } +return error;Just return the error directly from the loop and 0 if you make it through, then there is no need for the error initialization at the top either.
+1 for all of the above.
--NR
Re: [PATCH v1 3/7] xfs: Add realtime group grow tests
In reply to Christoph HellwigQuoted 11 lines · Christoph Hellwig
Quoted 4 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
Add unit tests to validate growth of realtime groups. This includes last rtgroup's partial growth case as well complete rtgroup addition.
Please tests these also with zoned rt devices. They are a bit special in that you can only grow (and in the future shrink) entire RTGs, and don't have the bitmap/summary. For some cases that might mean just not running them (if they grow inside a RTG), and for others they might need very minor tweaks.
Sure. I will update these tests once I am able to make the shrink work with the zoned devices. Thank you for pointing this out.
--NR
Quoted 1 line
Re: [RFC v1 4/4] xfs: Add support to shrink multiple empty realtime groups
In reply to Nirjhar Roy (IBM)Quoted 6 lines · Nirjhar Roy (IBM)
Quoted 2 lines
Highlevel note: this seems to only cover classic bitmap based RTGs, and not zoned ones. You might want to look into the latter because
Okay. So are you suggesting to add the shrink support only for the zoned devices or extend this patch set to cover shrink for the zoned devices too?
We should support all devices. But what I'm trying to say is the zoned version is both simpler, and will actually allow you to get to an actually useful fully feature shink much more easily.
Quoted 8 lines
Quoted 3 lines
they're actually much simpler, and with the zoned GC code we actually have a super nice way to move data out of the RTG. I'd be happy to supple you a code sniplet for the latter.
Sure, that would be super useful. Also, since I don't have much experience in zoned devices, can you please point to some relevant resources and recent zoned device related patchsets that can help me get started with?
You don't actually need a zoned device, you can just mkfs with "-r zoned=1" and get a internal rtdevice configuration. If you are interested in the actual hardware and ecosystem take a look at:
https://zonedstorage.io/
If at some point you're interested in hardware samples contact me offlist and I'll connect you to the right people.
Re: [RFC v1 4/4] xfs: Add support to shrink multiple empty realtime groups
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Nirjhar Roy (IBM)
Quoted 2 lines
Highlevel note: this seems to only cover classic bitmap based RTGs, and not zoned ones. You might want to look into the latter because
Okay. So are you suggesting to add the shrink support only for the zoned devices or extend this patch set to cover shrink for the zoned devices too?
We should support all devices. But what I'm trying to say is the zoned version is both simpler, and will actually allow you to get to an actually useful fully feature shink much more easily.
Okay. I will start taking a look at it.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
they're actually much simpler, and with the zoned GC code we actually have a super nice way to move data out of the RTG. I'd be happy to supple you a code sniplet for the latter.
Quoted 7 lines
Quoted 4 lines
Sure, that would be super useful. Also, since I don't have much experience in zoned devices, can you please point to some relevant resources and recent zoned device related patchsets that can help me get started with?
You don't actually need a zoned device, you can just mkfs with "-r zoned=1" and get a internal rtdevice configuration. If you are
Oh nice!.
Quoted 5 lines
interested in the actual hardware and ecosystem take a look at:
https://zonedstorage.io/If at some point you're interested in hardware samples contact me offlist and I'll connect you to the right people.
Thank you for the pointers.
--NR
Quoted 1 line
Re: [RFC v1 0/4] xfs: Add support to shrink multiple empty rtgroups
In reply to Nirjhar Roy (IBM)Quoted 37 lines · Nirjhar Roy (IBM)
From: "Nirjhar Roy (IBM)" <[email protected]>
This work is based on a previous RFC[1] by Gao Xiang and various ideas proposed by Dave Chinner in the RFC[1].
The patch begins with the re-introduction of some of the data structures that were removed, some code refactoring and finally the patch that implements the multi rtgroup shrink design. We can remove only those rtgroups which are empty.
For non-empty rtgroups the rtextents can migrated to other rtgroups (assuming the required amount of space is available).
I am working on the patch series [2](by Darrick and
Dave) for completion of the data block migration from the end of the filesystem for emptying an realtime group. That will be a future work after this.
The final patch has all the details including the definition of the terminologies and the overall design.The design is quite similar to the design of data AG removal implemented in the patch series[3] that I have posted earlier.
The reason for keeping [3] on hold and posting this patch series is that (based on the discussion in [5], [6]), realtime devices won't have any metadata/inodes and migrating data from the end of realtime devices will be simpler. On the other hand there are challenges in moving metadata from regular AGs especially inodes.Please note that I have added RBs from Darrick in patch 1 which was given in [4].
Instructions to test this patch:
$ Apply the patch for xfsprogs sent with this series and install it. $ mkfs.xfs -f -m metadir=1 -r rtdev=/dev/loop2,extsize=4096,rgcount=4,size=1G \ -d size=1G /dev/loop1 $ mount -o rtdev=/dev/loop2 /dev/loop1 /mnt/scratch $ # shrink by 1.5 rtgroups $ xfs_growfs -R $(( 65536 * 2 + 32768 )) /mnt1/scratch
Quoted 8 lines
I have also sent the tests.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-xfs/173568777852.2709794.6356870909327619205.stgit@frogsfrogsfrogs/
[3] https://lore.kernel.org/linux-xfs/[email protected]/
[4] https://lore.kernel.org/all/20250729202632.GF2672049@frogsfrogsfrogs/
[5] https://lore.kernel.org/linux-xfs/[email protected]/
[6] https://lore.kernel.org/linux-xfs/[email protected]/
Hi Christoph and Darrick,
I am preparing a v2 incorporating all the initial review feedback received so far. In parallel, I am looking into adding shrinkfs support for zoned XFS - however, it would be great if we can discuss the current design and get a full review of the changes so far to get this work (shrinkfs for non-zoned realtime groups) into proper shape for the upcoming merge window first and submit zoned XFS shrinkfs support separately afterward. Do you think that make sense?
--NR
Quoted 23 lines
Nirjhar Roy (IBM) (4):
xfs: Re-introducexg_active_wqfield in structxfs_group
xfs: Introducexfs_rtginodes_ensure_all()
xfs: Add a new state for shrinking
xfs: Add support to shrink multiple empty realtime groupsfs/xfs/libxfs/xfs_group.c | 18 +- fs/xfs/libxfs/xfs_group.h | 4 + fs/xfs/libxfs/xfs_rtgroup.c | 105 ++++- fs/xfs/libxfs/xfs_rtgroup.h | 31 ++ fs/xfs/xfs_buf_item_recover.c | 25 +- fs/xfs/xfs_extent_busy.c | 30 ++ fs/xfs/xfs_extent_busy.h | 2 + fs/xfs/xfs_inode.c | 8 +- fs/xfs/xfs_mount.h | 3 + fs/xfs/xfs_rtalloc.c | 824 +++++++++++++++++++++++++++++++++- fs/xfs/xfs_trans.c | 1 - 11 files changed, 1023 insertions(+), 28 deletions(-)--
2.43.5
Signature · 3 lines
Nirjhar Roy Linux Kernel Developer IBM, Bangalore
Re: [RFC v1 0/4] xfs: Add support to shrink multiple empty rtgroups
Quoted 7 lines · Nirjhar Roy (IBM)
I am preparing a v2 incorporating all the initial review feedback received so far. In parallel, I am looking into adding shrinkfs support for zoned XFS - however, it would be great if we can discuss the current design and get a full review of the changes so far to get this work (shrinkfs for non-zoned realtime groups) into proper shape for the upcoming merge window first and submit zoned XFS shrinkfs support separately afterward. Do you think that make sense?
What is your plan for actually migrating data out of the to be shrunk region, which is really the deal breaker to make this useful. Outside of that zone support should be really trivial as it just means doing a little less work.