xfsprogs: ports and new code for 6.16
PATCH v2 1/1 xfsprogs: ports and new code for 6.16
Hi everyone,
This is a collection of all the new code that I have for xfsprogs 6.16. First are the libxfs ports from Linux 6.16; after that come the userspace changes to report hardware atomic write capabilities and format a filesystem that will play nicely with the storage; and finally a patch to remove EXPERIMENTAL from xfs_scrub.
v2: add RVBs, fix a few comments as needed.
--D
PATCH 1/3 xfsprogs: new libxfs code from kernel 6.16
Hi all,
Port kernel libxfs code to userspace.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=libxfs-sync-6.16
---
Commits in this patchset:
* xfs: add helpers to compute transaction reservation for finishing intent items
* xfs: allow block allocator to take an alignment hint
* xfs: commit CoW-based atomic writes atomically
* libxfs: add helpers to compute log item overhead
* xfs: add xfs_calc_atomic_write_unit_max()
* xfs: allow sysadmins to specify a maximum atomic write limit at mount time
--- include/platform_defs.h | 14 ++ include/xfs_trace.h | 3 libxfs/defer_item.h | 14 ++ libxfs/xfs_bmap.h | 6 + libxfs/xfs_trans_resv.h | 25 +++ libxfs/defer_item.c | 51 +++++++ libxfs/xfs_bmap.c | 5 + libxfs/xfs_log_rlimit.c | 4 + libxfs/xfs_trans_resv.c | 339 +++++++++++++++++++++++++++++++++++++++++++---- 9 files changed, 429 insertions(+), 32 deletions(-)
PATCH 1/6 xfs: add helpers to compute transaction reservation for finishing intent items
From: Darrick J. Wong <[email protected]>
Source kernel commit: 805f89881252a9aee30799b8a395deec79c13414
In the transaction reservation code, hoist the logic that computes the reservation needed to finish one log intent item into separate helper functions. These will be used in subsequent patches to estimate the number of blocks that an online repair can commit to reaping in the same transaction as the change committing the new data structure.
--- libxfs/xfs_trans_resv.h | 18 +++++ libxfs/xfs_trans_resv.c | 165 ++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 152 insertions(+), 31 deletions(-)
libxfs/xfs_trans_resv.h +18 -0
@@ -98,6 +98,24 @@ struct xfs_trans_resv {void xfs_trans_resv_calc (struct xfs_mount *mp,struct xfs_trans_resv *resp);uint xfs_allocfree_block_count (struct xfs_mount *mp,uint num_ops); +unsigned int xfs_calc_finish_bui_reservation (struct xfs_mount *mp, +unsigned int nr_ops); + +unsigned int xfs_calc_finish_efi_reservation (struct xfs_mount *mp, +unsigned int nr_ops); +unsigned int xfs_calc_finish_rt_efi_reservation (struct xfs_mount *mp, +unsigned int nr_ops); + +unsigned int xfs_calc_finish_rui_reservation (struct xfs_mount *mp, +unsigned int nr_ops); +unsigned int xfs_calc_finish_rt_rui_reservation (struct xfs_mount *mp, +unsigned int nr_ops); + +unsigned int xfs_calc_finish_cui_reservation (struct xfs_mount *mp, +unsigned int nr_ops); +unsigned int xfs_calc_finish_rt_cui_reservation (struct xfs_mount *mp, +unsigned int nr_ops); +unsigned int xfs_calc_itruncate_reservation_minlogsize (struct xfs_mount *mp);unsigned int xfs_calc_write_reservation_minlogsize (struct xfs_mount *mp);unsigned int xfs_calc_qm_dqalloc_reservation_minlogsize (struct xfs_mount *mp);
libxfs/xfs_trans_resv.c +134 -31
208 lines not on this page · raw message
PATCH 2/6 xfs: allow block allocator to take an alignment hint
In reply to Darrick J. WongFrom: John Garry <[email protected]>
Source kernel commit: 6baf4cc47a741024d37e6149d5d035d3fc9ed1fe
Add a BMAPI flag to provide a hint to the block allocator to align extents according to the extszhint.
This will be useful for atomic writes to ensure that we are not being allocated extents which are not suitable (for atomic writes).
--- libxfs/xfs_bmap.h | 6 +++++- libxfs/xfs_bmap.c | 5 +++++ 2 files changed, 10 insertions(+), 1 deletion(-)
libxfs/xfs_bmap.h +5 -1
@@ -87,6 +87,9 @@ struct xfs_bmalloca {/* Do not update the rmap btree. Used for reconstructing bmbt from rmapbt. */ #define XFS_BMAPI_NORMAP (1u << 10) +/* Try to align allocations to the extent size hint */ +#define XFS_BMAPI_EXTSZALIGN (1u << 11) +#define XFS_BMAPI_FLAGS \ { XFS_BMAPI_ENTIRE, "ENTIRE" }, \ { XFS_BMAPI_METADATA, "METADATA" }, \
@@ -98,7 +101,8 @@ struct xfs_bmalloca { { XFS_BMAPI_REMAP, "REMAP" }, \ { XFS_BMAPI_COWFORK, "COWFORK" }, \ { XFS_BMAPI_NODISCARD, "NODISCARD" }, \ - { XFS_BMAPI_NORMAP, "NORMAP" } + { XFS_BMAPI_NORMAP, "NORMAP" },\ + { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" } static inline int xfs_bmapi_aflag(int w)
libxfs/xfs_bmap.c +5 -0
@@ -3306,6 +3306,11 @@ xfs_bmap_compute_alignments( align =xfs_get_cowextsz_hint (ap->ip); else if (ap->datatype & XFS_ALLOC_USERDATA) align =xfs_get_extsz_hint (ap->ip); + +/* Try to align start block to any minimum allocation alignment */ +if (align >1 && (ap->flags &XFS_BMAPI_EXTSZALIGN )) + args->alignment = align; +if (align) { if (xfs_bmap_extsize_align(mp, &ap->got, &ap->prev, align, 0, ap->eof, 0, ap->conv, &ap->offset,
PATCH 3/6 xfs: commit CoW-based atomic writes atomically
In reply to Darrick J. WongFrom: John Garry <[email protected]>
Source kernel commit: b1e09178b73adf10dc87fba9aee7787a7ad26874
When completing a CoW-based write, each extent range mapping update is covered by a separate transaction.
For a CoW-based atomic write, all mappings must be changed at once, so change to use a single transaction.
Note that there is a limit on the amount of log intent items which can be fit into a single transaction, but this is being ignored for now since the count of items for a typical atomic write would be much less than is typically supported. A typical atomic write would be expected to be 64KB or less, which means only 16 possible extents unmaps, which is quite small.
--- libxfs/xfs_trans_resv.h | 1 + libxfs/xfs_log_rlimit.c | 4 ++++ libxfs/xfs_trans_resv.c | 15 +++++++++++++++ 3 files changed, 20 insertions(+)
libxfs/xfs_trans_resv.h +1 -0
@@ -48,6 +48,7 @@ struct xfs_trans_resv {struct xfs_trans_res tr_qm_dqalloc;/* allocate quota on disk */ struct xfs_trans_res tr_sb;/* modify superblock */ struct xfs_trans_res tr_fsyncts;/* update timestamps on fsync */ +struct xfs_trans_res tr_atomic_ioend;/* untorn write completion */ };/* shorthand way of accessing reservation structure */
libxfs/xfs_log_rlimit.c +4 -0
@@ -91,6 +91,7 @@ xfs_log_calc_trans_resv_for_minlogblocks(*/ if (xfs_want_minlogsize_fixes (&mp->m_sb)) {xfs_trans_resv_calc (mp, resv); + resv->tr_atomic_ioend =M_RES (mp)->tr_atomic_ioend;return ; }
@@ -107,6 +108,9 @@ xfs_log_calc_trans_resv_for_minlogblocks(xfs_trans_resv_calc (mp, resv); +/* Copy the dynamic transaction reservation types from the running fs */ + resv->tr_atomic_ioend =M_RES (mp)->tr_atomic_ioend; +if (xfs_has_reflink (mp)) {/* * In the early days of reflink, typical log operation counts
libxfs/xfs_trans_resv.c +15 -0
@@ -1281,6 +1281,15 @@ xfs_calc_namespace_reservations( resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; } +STATIC void +xfs_calc_default_atomic_ioend_reservation ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +/* Pick a default that will scale reasonably for the log size. */ + resp->tr_atomic_ioend = resp->tr_itruncate; +} + void xfs_trans_resv_calc( struct xfs_mount *mp,
@@ -1375,4 +1384,10 @@ xfs_trans_resv_calc( resp->tr_itruncate.tr_logcount += logcount_adj; resp->tr_write.tr_logcount += logcount_adj; resp->tr_qm_dqalloc.tr_logcount += logcount_adj; + +/* +* Now that we've finished computing the static reservations, we can +* compute the dynamic reservation for atomic writes. +*/ +xfs_calc_default_atomic_ioend_reservation (mp, resp); }
PATCH 4/6 libxfs: add helpers to compute log item overhead
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add selected helpers to estimate the transaction reservation required to write various log intent and buffer items to the log. These helpers will be used by the online repair code for more precise estimations of how much work can be done in a single transaction.
--- libxfs/defer_item.h | 14 ++++++++++++++ libxfs/defer_item.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+)
libxfs/defer_item.h +14 -0
@@ -39,4 +39,18 @@ struct xfs_refcount_intent;void xfs_refcount_defer_add (struct xfs_trans *tp,struct xfs_refcount_intent *ri); +/* log intent size calculations */ + +unsigned int xfs_efi_log_space (unsigned int nr); +unsigned int xfs_efd_log_space (unsigned int nr); + +unsigned int xfs_rui_log_space (unsigned int nr); +unsigned int xfs_rud_log_space (void ); + +unsigned int xfs_bui_log_space (unsigned int nr); +unsigned int xfs_bud_log_space (void ); + +unsigned int xfs_cui_log_space (unsigned int nr); +unsigned int xfs_cud_log_space (void ); +#endif /* __LIBXFS_DEFER_ITEM_H_ */
libxfs/defer_item.c +51 -0
54 lines not on this page · raw message
Re: [PATCH 4/6] libxfs: add helpers to compute log item overhead
Quoted 8 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add selected helpers to estimate the transaction reservation required to write various log intent and buffer items to the log. These helpers will be used by the online repair code for more precise estimations of how much work can be done in a single transaction.
Signed-off-by "Darrick J. Wong" <[email protected]>
LGTM
Quoted 89 lines
--- libxfs/defer_item.h | 14 ++++++++++++++ libxfs/defer_item.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+)libxfs/defer_item.h +14
-0@@ -39,4 +39,18 @@ struct xfs_refcount_intent;void xfs_refcount_defer_add (struct xfs_trans *tp,struct xfs_refcount_intent *ri); +/* log intent size calculations */ + +unsigned int xfs_efi_log_space (unsigned int nr); +unsigned int xfs_efd_log_space (unsigned int nr); + +unsigned int xfs_rui_log_space (unsigned int nr); +unsigned int xfs_rud_log_space (void ); + +unsigned int xfs_bui_log_space (unsigned int nr); +unsigned int xfs_bud_log_space (void ); + +unsigned int xfs_cui_log_space (unsigned int nr); +unsigned int xfs_cud_log_space (void ); +#endif /* __LIBXFS_DEFER_ITEM_H_ */ libxfs/defer_item.c +51
-0@@ -942,3 +942,54 @@ const struct xfs_defer_op_type xfs_exchmaps_defer_type = { .finish_item = xfs_exchmaps_finish_item, .cancel_item = xfs_exchmaps_cancel_item, }; + +/* log intent size calculations */ + +static inline unsigned int +xlog_item_space ( +unsigned int niovecs, +unsigned int nbytes) +{ + nbytes += niovecs * (sizeof (uint64_t ) +sizeof (struct xlog_op_header )); +return round_up (nbytes,sizeof (uint64_t )); +} + +unsigned int xfs_efi_log_space (unsigned int nr) +{ +return xlog_item_space (1 ,xfs_efi_log_format_sizeof (nr)); +} + +unsigned int xfs_efd_log_space (unsigned int nr) +{ +return xlog_item_space (1 ,xfs_efd_log_format_sizeof (nr)); +} + +unsigned int xfs_rui_log_space (unsigned int nr) +{ +return xlog_item_space (1 ,xfs_rui_log_format_sizeof (nr)); +} + +unsigned int xfs_rud_log_space (void ) +{ +return xlog_item_space (1 ,sizeof (struct xfs_rud_log_format )); +} + +unsigned int xfs_bui_log_space (unsigned int nr) +{ +return xlog_item_space (1 ,xfs_bui_log_format_sizeof (nr)); +} + +unsigned int xfs_bud_log_space (void ) +{ +return xlog_item_space (1 ,sizeof (struct xfs_bud_log_format )); +} + +unsigned int xfs_cui_log_space (unsigned int nr) +{ +return xlog_item_space (1 ,xfs_cui_log_format_sizeof (nr)); +} + +unsigned int xfs_cud_log_space (void ) +{ +return xlog_item_space (1 ,sizeof (struct xfs_cud_log_format )); +}
Signature · 1 line
- Andrey
PATCH 5/6 xfs: add xfs_calc_atomic_write_unit_max()
In reply to Darrick J. WongFrom: John Garry <[email protected]>
Source kernel commit: 0c438dcc31504bf4f50b20dc52f8f5ca7fab53e2
Now that CoW-based atomic writes are supported, update the max size of an atomic write for the data device.
The limit of a CoW-based atomic write will be the limit of the number of logitems which can fit into a single transaction.
In addition, the max atomic write size needs to be aligned to the agsize. Limit the size of atomic writes to the greatest power-of-two factor of the agsize so that allocations for an atomic write will always be aligned compatibly with the alignment requirements of the storage.
Function xfs_atomic_write_logitems() is added to find the limit the number of log items which can fit in a single transaction.
Amend the max atomic write computation to create a new transaction reservation type, and compute the maximum size of an atomic write completion (in fsblocks) based on this new transaction reservation. Initially, tr_atomic_write is a clone of tr_itruncate, which provides a reasonable level of parallelism. In the next patch, we'll add a mount option so that sysadmins can configure their own limits.
[djwong: use a new reservation type for atomic write ioends, refactor group limit calculations]
--- include/xfs_trace.h | 2 + libxfs/xfs_trans_resv.h | 2 + libxfs/xfs_trans_resv.c | 90 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+)
include/xfs_trace.h +2 -0
@@ -64,6 +64,8 @@#define trace_xfs_attr_rmtval_alloc (...) ((void) 0)#define trace_xfs_attr_rmtval_remove_return (...) ((void) 0) +#define trace_xfs_calc_max_atomic_write_fsblocks (...) ((void) 0) +#define trace_xfs_log_recover_item_add_cont (a,b,c,d) ((void) 0)#define trace_xfs_log_recover_item_add (a,b,c,d) ((void) 0)
libxfs/xfs_trans_resv.h +2 -0
@@ -121,4 +121,6 @@ unsigned int xfs_calc_itruncate_reservation_minlogsize(struct xfs_mount *mp);unsigned int xfs_calc_write_reservation_minlogsize (struct xfs_mount *mp);unsigned int xfs_calc_qm_dqalloc_reservation_minlogsize (struct xfs_mount *mp); +xfs_extlen_t xfs_calc_max_atomic_write_fsblocks (struct xfs_mount *mp); +#endif /* __XFS_TRANS_RESV_H__ */
libxfs/xfs_trans_resv.c +90 -0
99 lines not on this page · raw message
PATCH 6/6 xfs: allow sysadmins to specify a maximum atomic write limit at mount time
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Source kernel commit: 4528b9052731f14c1a9be16b98e33c9401e6d1bc
Introduce a mount option to allow sysadmins to specify the maximum size of an atomic write. If the filesystem can work with the supplied value, that becomes the new guaranteed maximum.
The value mustn't be too big for the existing filesystem geometry (max write size, max AG/rtgroup size). We dynamically recompute the tr_atomic_write transaction reservation based on the given block size, check that the current log size isn't less than the new minimum log size constraints, and set a new maximum.
The actual software atomic write max is still computed based off of tr_atomic_ioend the same way it has for the past few commits. Note also that xfs_calc_atomic_write_log_geometry is non-static because mkfs will need that.
--- include/platform_defs.h | 14 ++++++++++ include/xfs_trace.h | 1 + libxfs/xfs_trans_resv.h | 4 +++ libxfs/xfs_trans_resv.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 88 insertions(+)
include/platform_defs.h +14 -0
@@ -261,6 +261,20 @@ static inline bool __must_check __must_check_overflow(bool overflow)__builtin_add_overflow (__a, __b, __d); \ })) +/** +* check_mul_overflow() - Calculate multiplication with overflow checking +* @a: first factor +* @b: second factor +* @d: pointer to store product +* +* Returns true on wrap-around, false otherwise. +* +* *@d holds the results of the attempted multiplication, regardless of whether +* wrap-around occurred. +*/ +#define check_mul_overflow (a, b, d) \ + __must_check_overflow(__builtin_mul_overflow(a, b, d)) +/** * abs_diff - return absolute value of the difference between the arguments * @a: the first argument
include/xfs_trace.h +1 -0
@@ -65,6 +65,7 @@#define trace_xfs_attr_rmtval_remove_return (...) ((void) 0)#define trace_xfs_calc_max_atomic_write_fsblocks (...) ((void) 0) +#define trace_xfs_calc_max_atomic_write_log_geometry (...) ((void) 0)#define trace_xfs_log_recover_item_add_cont (a,b,c,d) ((void) 0)#define trace_xfs_log_recover_item_add (a,b,c,d) ((void) 0)
libxfs/xfs_trans_resv.h +4 -0
@@ -122,5 +122,9 @@ unsigned int xfs_calc_write_reservation_minlogsize(struct xfs_mount *mp);unsigned int xfs_calc_qm_dqalloc_reservation_minlogsize (struct xfs_mount *mp);xfs_extlen_t xfs_calc_max_atomic_write_fsblocks (struct xfs_mount *mp); +xfs_extlen_t xfs_calc_atomic_write_log_geometry (struct xfs_mount *mp, +xfs_extlen_t blockcount,unsigned int *new_logres); +int xfs_calc_atomic_write_reservation (struct xfs_mount *mp, +xfs_extlen_t blockcount);#endif /* __XFS_TRANS_RESV_H__ */
libxfs/xfs_trans_resv.c +69 -0
72 lines not on this page · raw message
PATCH 2/3 xfsprogs: atomic writes
In reply to Darrick J. WongHi all,
Utility updates for dumping atomic writes configuration and formatting filesystems to take advantage of the new functionality.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=atomic-writes-6.16
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=atomic-writes-6.16
---
Commits in this patchset:
* libfrog: move statx.h from io/ to libfrog/
* xfs_db: create an untorn_max subcommand
* xfs_io: dump new atomic_write_unit_max_opt statx field
* mkfs: don't complain about overly large auto-detected log stripe units
* mkfs: autodetect log stripe unit for external log devices
* mkfs: try to align AG size based on atomic write capabilities
* mkfs: allow users to configure the desired maximum atomic write size
--- include/bitops.h | 12 ++ include/libxfs.h | 1 libfrog/statx.h | 23 ++++ libxfs/libxfs_api_defs.h | 5 + libxfs/topology.h | 6 + db/logformat.c | 129 ++++++++++++++++++++++++ io/stat.c | 21 +--- libfrog/Makefile | 1 libxfs/topology.c | 36 +++++++ m4/package_libcdev.m4 | 2 man/man8/mkfs.xfs.8.in | 7 + man/man8/xfs_db.8 | 10 ++ mkfs/xfs_mkfs.c | 251 +++++++++++++++++++++++++++++++++++++++++++++- 13 files changed, 474 insertions(+), 30 deletions(-) rename io/statx.h => libfrog/statx.h (94%)
PATCH 1/7 libfrog: move statx.h from io/ to libfrog/
From: Darrick J. Wong <[email protected]>
Move this header file so we can use the declaration and wrappers in other parts of xfsprogs.
--- libfrog/statx.h | 17 +++++++++++++++++ io/stat.c | 20 ++------------------ libfrog/Makefile | 1 + 3 files changed, 20 insertions(+), 18 deletions(-) rename io/statx.h => libfrog/statx.h (96%)
libfrog/statx.h renamed from io/statx.h +17 -0
@@ -146,7 +146,24 @@ struct statx {__u64 __spare3[9 ];/* Spare space for future expansion */ /* 0x100 */ }; + +static inline ssize_t +statx ( +int dfd, +const char *filename, +unsigned int flags, +unsigned int mask, +struct statx *buffer) +{ +#ifdef __NR_statx +return syscall (__NR_statx, dfd, filename, flags, mask, buffer); +#else + errno =ENOSYS ; +return -1 ;#endif +} + +#endif /* OVERRIDE_SYSTEM_STATX */ #ifndef STATX_TYPE /*
io/stat.c +2 -18
@@ -14,7 +14,7 @@#include "input.h" #include "init.h" #include "io.h" -#include "statx.h" +#include "libfrog/statx.h" #include "libxfs.h" #include "libfrog/logging.h" #include "libfrog/fsgeom.h"
@@ -305,22 +305,6 @@ statfs_f(return 0 ; } -static ssize_t -_statx ( -int dfd, -const char *filename, -unsigned int flags, -unsigned int mask, -struct statx *buffer) -{ -#ifdef __NR_statx -return syscall (__NR_statx, dfd, filename, flags, mask, buffer); -#else - errno =ENOSYS ; -return -1 ; -#endif -} -struct statx_masks {const char *name;unsigned int mask;
@@ -525,7 +509,7 @@ statx_f(return command_usage (&statx_cmd);memset (&stx,0xbf ,sizeof (stx)); -if (_statx (file->fd,"" , atflag |AT_EMPTY_PATH , mask, &stx) <0 ) { +if (statx (file->fd,"" , atflag |AT_EMPTY_PATH , mask, &stx) <0 ) {perror ("statx" ); exitcode =1 ;return 0 ;
libfrog/Makefile +1 -0
@@ -62,6 +62,7 @@ ptvar.h \ radix-tree.h \ randbytes.h \ scrub.h \ +statx.h \ workqueue.h GETTEXT_PY = \
PATCH 2/7 xfs_db: create an untorn_max subcommand
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a debugger command to compute the either the logres needed to perform an untorn cow write completion for a given number of blocks; or the number of blocks that can be completed given a log reservation.
--- include/libxfs.h | 1 libxfs/libxfs_api_defs.h | 4 + db/logformat.c | 129 ++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_db.8 | 10 ++++ 4 files changed, 144 insertions(+)
include/libxfs.h +1 -0
@@ -102,6 +102,7 @@ struct iomap;#include "xfs_rtbitmap.h" #include "xfs_rtrmap_btree.h" #include "xfs_ag_resv.h" +#include "defer_item.h" #ifndef ARRAY_SIZE #define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0]))
libxfs/libxfs_api_defs.h +4 -0
@@ -108,6 +108,10 @@#define xfs_bunmapi libxfs_bunmapi#define xfs_bwrite libxfs_bwrite#define xfs_calc_dquots_per_chunk libxfs_calc_dquots_per_chunk +#define xfs_calc_finish_bui_reservation libxfs_calc_finish_bui_reservation +#define xfs_calc_finish_cui_reservation libxfs_calc_finish_cui_reservation +#define xfs_calc_finish_efi_reservation libxfs_calc_finish_efi_reservation +#define xfs_calc_finish_rui_reservation libxfs_calc_finish_rui_reservation#define xfs_cntbt_init_cursor libxfs_cntbt_init_cursor#define xfs_compute_rextslog libxfs_compute_rextslog#define xfs_compute_rgblklog libxfs_compute_rgblklog
db/logformat.c +129 -0
137 lines not on this page · raw message
man/man8/xfs_db.8 +10 -0
@@ -1366,6 +1366,16 @@ .SH COMMANDS .IR name . The file being targetted will not be put on the iunlink list. .TP +.BI "untorn_write_max [\-b " blockcount "|\-l " logres "]" +If +.B -l +is specified, compute the maximum (in fsblocks) untorn write that we can +emulate with copy on write given a log reservation size (in bytes). +If +.B -b +is specified, compute the log reservation size that would be needed to +emulate an untorn write of the given number of fsblocks. +.TP .BI "uuid [" uuid " | " generate " | " rewrite " | " restore ] Set the filesystem universally unique identifier (UUID). The filesystem UUID can be used by
PATCH 3/7 xfs_io: dump new atomic_write_unit_max_opt statx field
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Dump the new atomic writes statx field that's being submitted for 6.16.
--- libfrog/statx.h | 6 +++++- io/stat.c | 1 + m4/package_libcdev.m4 | 2 +- 3 files changed, 7 insertions(+), 2 deletions(-)
libfrog/statx.h +5 -1
@@ -143,7 +143,11 @@ struct statx {__u32 stx_dio_read_offset_align;/* 0xb8 */ -__u64 __spare3[9 ];/* Spare space for future expansion */ +/* Optimised max atomic write unit in bytes */ +__u32 stx_atomic_write_unit_max_opt; +__u32 __spare2[1 ]; +/* 0xc0 */ +__u64 __spare3[8 ];/* Spare space for future expansion */ /* 0x100 */ };
io/stat.c +1 -0
@@ -396,6 +396,7 @@ dump_raw_statx(struct statx *stx)printf ("stat.atomic_write_unit_max = %u\n" , stx->stx_atomic_write_unit_max);printf ("stat.atomic_write_segments_max = %u\n" , stx->stx_atomic_write_segments_max);printf ("stat.dio_read_offset_align = %u\n" , stx->stx_dio_read_offset_align); +printf ("stat.atomic_write_unit_max_opt = %u\n" , stx->stx_atomic_write_unit_max_opt);return 0 ; }
m4/package_libcdev.m4 +1 -1
@@ -126,7 +126,7 @@ AC_DEFUN([AC_NEED_INTERNAL_FSCRYPT_POLICY_V2], AC_DEFUN([AC_NEED_INTERNAL_STATX], [ AC_CHECK_TYPE(struct statx, [ - AC_CHECK_MEMBER(struct statx.stx_dio_read_offset_align, + AC_CHECK_MEMBER(struct statx.stx_atomic_write_unit_max_opt, , need_internal_statx=yes, [#include <linux/stat.h>]
PATCH 4/7 mkfs: don't complain about overly large auto-detected log stripe units
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If mkfs declines to apply what it thinks is an overly large data device stripe unit to the log device, it should only log a message about that if the lsunit parameter was actually supplied by the caller. It should not do that when the lsunit was autodetected from the block devices.
The cli parameters are zero-initialized in main and always have been.
--- mkfs/xfs_mkfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mkfs/xfs_mkfs.c +1 -1
@@ -3629,7 +3629,7 @@ _("log stripe unit (%d) must be a multiple of the block size (%d)\n"),if (cfg->sb_feat.log_version ==2 && cfg->lsunit * cfg->blocksize >256 *1024 ) {/* Warn only if specified on commandline */ -if (cli->lsu || cli->lsunit !=-1 ) { +if (cli->lsu || cli->lsunit) { fprintf(stderr, _("log stripe unit (%d bytes) is too large (maximum is 256KiB)\n" "log stripe unit adjusted to 32KiB\n"),
PATCH 5/7 mkfs: autodetect log stripe unit for external log devices
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If we're using an external log device and the caller doesn't give us a lsunit, use the block device geometry (if present) to set it.
--- mkfs/xfs_mkfs.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
mkfs/xfs_mkfs.c +6 -1
@@ -3615,7 +3615,8 @@ _("log stripe unit (%d) must be a multiple of the block size (%d)\n"), }/* -* check that log sunit is modulo fsblksize or default it to dsunit. +* check that log sunit is modulo fsblksize; default it to dsunit for +* an internal log; or the log device stripe unit if it's external. */ if (lsunit) {/* convert from 512 byte blocks to fs blocks */
@@ -3624,6 +3625,10 @@ _("log stripe unit (%d) must be a multiple of the block size (%d)\n"), cfg->loginternal && cfg->dsunit) {/* lsunit and dsunit now in fs blocks */ cfg->lsunit = cfg->dsunit; + } else if (cfg->sb_feat.log_version == 2 && + !cfg->loginternal) { +/* use the external log device properties */ + cfg->lsunit =DTOBT (ft->log.sunit, cfg->blocklog); } if (cfg->sb_feat.log_version == 2 &&
PATCH 6/7 mkfs: try to align AG size based on atomic write capabilities
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Try to align the AG size to the maximum hardware atomic write unit so that we can give users maximum flexibility in choosing an RWF_ATOMIC write size.
--- libxfs/topology.h | 6 ++++-- libxfs/topology.c | 36 ++++++++++++++++++++++++++++++++++++ mkfs/xfs_mkfs.c | 48 +++++++++++++++++++++++++++++++++++++++++++----- 3 files changed, 83 insertions(+), 7 deletions(-)
libxfs/topology.h +4 -2
@@ -13,8 +13,10 @@struct device_topology {int logical_sector_size;/* logical sector size */ int physical_sector_size;/* physical sector size */ -int sunit;/* stripe unit */ -int swidth;/* stripe width */ +int sunit;/* stripe unit */ +int swidth;/* stripe width */ +int awu_min;/* min atomic write unit in bbcounts */ +int awu_max;/* max atomic write unit in bbcounts */ };struct fs_topology {
libxfs/topology.c +36 -0
@@ -4,11 +4,18 @@* All Rights Reserved. */ +#ifdef OVERRIDE_SYSTEM_STATX +#define statx sys_statx +#endif +#include <fcntl.h> +#include <sys/stat.h> +#include "libxfs_priv.h" #include "libxcmd.h" #include <blkid/blkid.h> #include "xfs_multidisk.h" #include "libfrog/platform.h" +#include "libfrog/statx.h" #define TERABYTES (count, blog) ((uint64_t)(count) << (40 - (blog)))#define GIGABYTES (count, blog) ((uint64_t)(count) << (30 - (blog)))
@@ -278,6 +285,34 @@ blkid_get_topology( device); } +static void +get_hw_atomic_writes_topology ( +struct libxfs_dev *dev, +struct device_topology *dt) +{ +struct statx sx; +int fd; +int ret; + + fd =open (dev->name,O_RDONLY ); +if (fd <0 ) +return ; + + ret =statx (fd,"" ,AT_EMPTY_PATH ,STATX_WRITE_ATOMIC , &sx); +if (ret) + goto out_close; + +if (!(sx.stx_mask &STATX_WRITE_ATOMIC )) + goto out_close; + + dt->awu_min = sx.stx_atomic_write_unit_min >>9 ; + dt->awu_max =max (sx.stx_atomic_write_unit_max_opt, + sx.stx_atomic_write_unit_max) >>9 ; + +out_close: +close (fd); +} + static void get_device_topology( struct libxfs_dev *dev,
@@ -316,6 +351,7 @@ get_device_topology( } }else {blkid_get_topology (dev->name, dt, force_overwrite); +get_hw_atomic_writes_topology (dev, dt); }ASSERT (dt->logical_sector_size);
mkfs/xfs_mkfs.c +43 -5
79 lines not on this page · raw message
PATCH 7/7 mkfs: allow users to configure the desired maximum atomic write size
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Allow callers of mkfs.xfs to specify a desired maximum atomic write size. This value will cause the log size to be adjusted to support software atomic writes, and the AG size to be aligned to support hardware atomic writes.
--- include/bitops.h | 12 +++ libxfs/libxfs_api_defs.h | 1 man/man8/mkfs.xfs.8.in | 7 ++ mkfs/xfs_mkfs.c | 194 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 213 insertions(+), 1 deletion(-)
include/bitops.h +12 -0
@@ -113,4 +113,16 @@ static inline int lowbit64(uint64_t v) return n - 1; } +/** +* __rounddown_pow_of_two() - round down to nearest power of two +* @n: value to round down +*/ +static inline __attribute__((const)) +unsigned long __rounddown_pow_of_two (unsigned long n) +{ +return 1UL << (fls_long (n) -1 ); +} + +#define rounddown_pow_of_two (n) __rounddown_pow_of_two(n) +#endif
libxfs/libxfs_api_defs.h +1 -0
@@ -107,6 +107,7 @@#define xfs_buftarg_drain libxfs_buftarg_drain#define xfs_bunmapi libxfs_bunmapi#define xfs_bwrite libxfs_bwrite +#define xfs_calc_atomic_write_log_geometry libxfs_calc_atomic_write_log_geometry#define xfs_calc_dquots_per_chunk libxfs_calc_dquots_per_chunk#define xfs_calc_finish_bui_reservation libxfs_calc_finish_bui_reservation#define xfs_calc_finish_cui_reservation libxfs_calc_finish_cui_reservation
man/man8/mkfs.xfs.8.in +7 -0
@@ -742,6 +742,13 @@ .SH OPTIONS directories, symbolic links, and realtime metadata files. This feature is disabled by default. This feature is only available for filesystems formatted with -m crc=1. +.TP +.BI max_atomic_write[= value] +When enabled, application programs can use the RWF_ATOMIC write flag to +persist changes of up to this size without tearing. +The default is chosen to allow a reasonable amount of scalability. +This value must also be passed via mount option. +This feature is only available for filesystems formatted with reflink. .RE .PP .PD 0
mkfs/xfs_mkfs.c +193 -1
266 lines not on this page · raw message
PATCH 3/3 xfs_scrub: drop EXPERIMENTAL warning
In reply to Darrick J. WongHi all,
Drop the EXPERIMENTAL warning on xfs_scrub now that we've done so for the kernel.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-6.16
---
Commits in this patchset:
* xfs_scrub: remove EXPERIMENTAL warnings
--- man/man8/xfs_scrub.8 | 6 ------ scrub/xfs_scrub.c | 3 --- 2 files changed, 9 deletions(-)
PATCH 1/1 xfs_scrub: remove EXPERIMENTAL warnings
From: Darrick J. Wong <[email protected]>
The kernel code for online fsck has been stable for a year, and there haven't been any major changes to the program in quite some time, so let's drop the experimental warnings.
--- man/man8/xfs_scrub.8 | 6 ------ scrub/xfs_scrub.c | 3 --- 2 files changed, 9 deletions(-)
man/man8/xfs_scrub.8 +0 -6
@@ -13,12 +13,6 @@ .SH DESCRIPTION .B xfs_scrub attempts to check and repair all metadata in a mounted XFS filesystem. .PP -.B WARNING! -This program is -.BR EXPERIMENTAL "," -which means that its behavior and interface -could change at any time! -.PP .B xfs_scrub asks the kernel to scrub all metadata objects in the filesystem. Metadata records are scanned for obviously bad values and then
scrub/xfs_scrub.c +0 -3
@@ -730,9 +730,6 @@ main(hist_init (&ctx.datadev_hist);hist_init (&ctx.rtdev_hist); -fprintf (stdout,"EXPERIMENTAL xfs_scrub program in use! Use at your own risk!\n" ); -fflush (stdout); - progname =basename (argv[0 ]);setlocale (LC_ALL ,"" );bindtextdomain (PACKAGE ,LOCALEDIR );