Skip to contentSkip to conversation

xfsprogs: ports and new code for 6.16

PATCH New Darrick J. Wong 2 participants 19 messages Jul 15, 2025 – Jul 18, 2025

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 xfsprogs: ports and new code for 6.16

Hi everyone,

This is a collection of all the new code that I have for xfsprogs 6.16. First are the libxfs ports from Linux 6.16; after that come the userspace changes to report hardware atomic write capabilities and format a filesystem that will play nicely with the storage; and finally a patch to remove EXPERIMENTAL from xfs_scrub.

v2: add RVBs, fix a few comments as needed.

--D

#1 lore raw

PATCH 1/3 xfsprogs: new libxfs code from kernel 6.16

Hi all,

Port kernel libxfs code to userspace.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

Comments and questions are, as always, welcome.

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=libxfs-sync-6.16
---
Commits in this patchset:
* xfs: add helpers to compute transaction reservation for finishing intent items
* xfs: allow block allocator to take an alignment hint
* xfs: commit CoW-based atomic writes atomically
* libxfs: add helpers to compute log item overhead
* xfs: add xfs_calc_atomic_write_unit_max()
* xfs: allow sysadmins to specify a maximum atomic write limit at mount time

---
 include/platform_defs.h |   14 ++
 include/xfs_trace.h     |    3 
 libxfs/defer_item.h     |   14 ++
 libxfs/xfs_bmap.h       |    6 +
 libxfs/xfs_trans_resv.h |   25 +++
 libxfs/defer_item.c     |   51 +++++++
 libxfs/xfs_bmap.c       |    5 +
 libxfs/xfs_log_rlimit.c |    4 +
 libxfs/xfs_trans_resv.c |  339 +++++++++++++++++++++++++++++++++++++++++++----
 9 files changed, 429 insertions(+), 32 deletions(-)
#2 lore raw

PATCH 1/6 xfs: add helpers to compute transaction reservation for finishing intent items

From: Darrick J. Wong <[email protected]>

Source kernel commit: 805f89881252a9aee30799b8a395deec79c13414

In the transaction reservation code, hoist the logic that computes the reservation needed to finish one log intent item into separate helper functions. These will be used in subsequent patches to estimate the number of blocks that an online repair can commit to reaping in the same transaction as the change committing the new data structure.

---
 libxfs/xfs_trans_resv.h |   18 +++++
 libxfs/xfs_trans_resv.c |  165 ++++++++++++++++++++++++++++++++++++++---------
 2 files changed, 152 insertions(+), 31 deletions(-)
libxfs/xfs_trans_resv.h +18 -0
index 0554b9d775d269..d9d0032cbbc5d4 100644
@@ -98,6 +98,24 @@ struct xfs_trans_resv {
 void xfs_trans_resv_calc(struct xfs_mount *mp, struct xfs_trans_resv *resp);
 uint xfs_allocfree_block_count(struct xfs_mount *mp, uint num_ops);
 
+unsigned int xfs_calc_finish_bui_reservation(struct xfs_mount *mp,
+		unsigned int nr_ops);
+
+unsigned int xfs_calc_finish_efi_reservation(struct xfs_mount *mp,
+		unsigned int nr_ops);
+unsigned int xfs_calc_finish_rt_efi_reservation(struct xfs_mount *mp,
+		unsigned int nr_ops);
+
+unsigned int xfs_calc_finish_rui_reservation(struct xfs_mount *mp,
+		unsigned int nr_ops);
+unsigned int xfs_calc_finish_rt_rui_reservation(struct xfs_mount *mp,
+		unsigned int nr_ops);
+
+unsigned int xfs_calc_finish_cui_reservation(struct xfs_mount *mp,
+		unsigned int nr_ops);
+unsigned int xfs_calc_finish_rt_cui_reservation(struct xfs_mount *mp,
+		unsigned int nr_ops);
+
 unsigned int xfs_calc_itruncate_reservation_minlogsize(struct xfs_mount *mp);
 unsigned int xfs_calc_write_reservation_minlogsize(struct xfs_mount *mp);
 unsigned int xfs_calc_qm_dqalloc_reservation_minlogsize(struct xfs_mount *mp);
libxfs/xfs_trans_resv.c +134 -31

208 lines not on this page · raw message

#3 lore raw

PATCH 2/6 xfs: allow block allocator to take an alignment hint

In reply to Darrick J. Wong

From: John Garry <[email protected]>

Source kernel commit: 6baf4cc47a741024d37e6149d5d035d3fc9ed1fe

Add a BMAPI flag to provide a hint to the block allocator to align extents according to the extszhint.

This will be useful for atomic writes to ensure that we are not being allocated extents which are not suitable (for atomic writes).

---
 libxfs/xfs_bmap.h |    6 +++++-
 libxfs/xfs_bmap.c |    5 +++++
 2 files changed, 10 insertions(+), 1 deletion(-)
libxfs/xfs_bmap.h +5 -1
index b4d9c6e0f3f9b6..d5f2729305fada 100644
@@ -87,6 +87,9 @@ struct xfs_bmalloca {
 /* Do not update the rmap btree.  Used for reconstructing bmbt from rmapbt. */
 #define XFS_BMAPI_NORMAP	(1u << 10)
 
+/* Try to align allocations to the extent size hint */
+#define XFS_BMAPI_EXTSZALIGN	(1u << 11)
+
 #define XFS_BMAPI_FLAGS \
 	{ XFS_BMAPI_ENTIRE,	"ENTIRE" }, \
 	{ XFS_BMAPI_METADATA,	"METADATA" }, \
@@ -98,7 +101,8 @@ struct xfs_bmalloca {
 	{ XFS_BMAPI_REMAP,	"REMAP" }, \
 	{ XFS_BMAPI_COWFORK,	"COWFORK" }, \
 	{ XFS_BMAPI_NODISCARD,	"NODISCARD" }, \
-	{ XFS_BMAPI_NORMAP,	"NORMAP" }
+	{ XFS_BMAPI_NORMAP,	"NORMAP" },\
+	{ XFS_BMAPI_EXTSZALIGN,	"EXTSZALIGN" }
 
 
 static inline int xfs_bmapi_aflag(int w)
libxfs/xfs_bmap.c +5 -0
index 3cb47c3c8707db..99f5e6f9d545a4 100644
@@ -3306,6 +3306,11 @@ xfs_bmap_compute_alignments(
 		align = xfs_get_cowextsz_hint(ap->ip);
 	else if (ap->datatype & XFS_ALLOC_USERDATA)
 		align = xfs_get_extsz_hint(ap->ip);
+
+	/* Try to align start block to any minimum allocation alignment */
+	if (align > 1 && (ap->flags & XFS_BMAPI_EXTSZALIGN))
+		args->alignment = align;
+
 	if (align) {
 		if (xfs_bmap_extsize_align(mp, &ap->got, &ap->prev, align, 0,
 					ap->eof, 0, ap->conv, &ap->offset,
#4 lore raw

PATCH 3/6 xfs: commit CoW-based atomic writes atomically

In reply to Darrick J. Wong

From: John Garry <[email protected]>

Source kernel commit: b1e09178b73adf10dc87fba9aee7787a7ad26874

When completing a CoW-based write, each extent range mapping update is covered by a separate transaction.

For a CoW-based atomic write, all mappings must be changed at once, so change to use a single transaction.

Note that there is a limit on the amount of log intent items which can be fit into a single transaction, but this is being ignored for now since the count of items for a typical atomic write would be much less than is typically supported. A typical atomic write would be expected to be 64KB or less, which means only 16 possible extents unmaps, which is quite small.

---
 libxfs/xfs_trans_resv.h |    1 +
 libxfs/xfs_log_rlimit.c |    4 ++++
 libxfs/xfs_trans_resv.c |   15 +++++++++++++++
 3 files changed, 20 insertions(+)
libxfs/xfs_trans_resv.h +1 -0
index d9d0032cbbc5d4..670045d417a65f 100644
@@ -48,6 +48,7 @@ struct xfs_trans_resv {
 	struct xfs_trans_res	tr_qm_dqalloc;	/* allocate quota on disk */
 	struct xfs_trans_res	tr_sb;		/* modify superblock */
 	struct xfs_trans_res	tr_fsyncts;	/* update timestamps on fsync */
+	struct xfs_trans_res	tr_atomic_ioend; /* untorn write completion */
 };
 
 /* shorthand way of accessing reservation structure */
libxfs/xfs_log_rlimit.c +4 -0
index 246d5f486d024a..2b9047f5ffb58b 100644
@@ -91,6 +91,7 @@ xfs_log_calc_trans_resv_for_minlogblocks(
 	 */
 	if (xfs_want_minlogsize_fixes(&mp->m_sb)) {
 		xfs_trans_resv_calc(mp, resv);
+		resv->tr_atomic_ioend = M_RES(mp)->tr_atomic_ioend;
 		return;
 	}
 
@@ -107,6 +108,9 @@ xfs_log_calc_trans_resv_for_minlogblocks(
 
 	xfs_trans_resv_calc(mp, resv);
 
+	/* Copy the dynamic transaction reservation types from the running fs */
+	resv->tr_atomic_ioend = M_RES(mp)->tr_atomic_ioend;
+
 	if (xfs_has_reflink(mp)) {
 		/*
 		 * In the early days of reflink, typical log operation counts
libxfs/xfs_trans_resv.c +15 -0
index 0a843deb50a118..cf735f946c8ac7 100644
@@ -1281,6 +1281,15 @@ xfs_calc_namespace_reservations(
 	resp->tr_mkdir.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
 }
 
+STATIC void
+xfs_calc_default_atomic_ioend_reservation(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	/* Pick a default that will scale reasonably for the log size. */
+	resp->tr_atomic_ioend = resp->tr_itruncate;
+}
+
 void
 xfs_trans_resv_calc(
 	struct xfs_mount	*mp,
@@ -1375,4 +1384,10 @@ xfs_trans_resv_calc(
 	resp->tr_itruncate.tr_logcount += logcount_adj;
 	resp->tr_write.tr_logcount += logcount_adj;
 	resp->tr_qm_dqalloc.tr_logcount += logcount_adj;
+
+	/*
+	 * Now that we've finished computing the static reservations, we can
+	 * compute the dynamic reservation for atomic writes.
+	 */
+	xfs_calc_default_atomic_ioend_reservation(mp, resp);
 }
#5 lore raw

PATCH 4/6 libxfs: add helpers to compute log item overhead

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add selected helpers to estimate the transaction reservation required to write various log intent and buffer items to the log. These helpers will be used by the online repair code for more precise estimations of how much work can be done in a single transaction.

---
 libxfs/defer_item.h |   14 ++++++++++++++
 libxfs/defer_item.c |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
libxfs/defer_item.h +14 -0
index 93cf1eed58a382..325a6f7b2dcbce 100644
@@ -39,4 +39,18 @@ struct xfs_refcount_intent;
 void xfs_refcount_defer_add(struct xfs_trans *tp,
 		struct xfs_refcount_intent *ri);
 
+/* log intent size calculations */
+
+unsigned int xfs_efi_log_space(unsigned int nr);
+unsigned int xfs_efd_log_space(unsigned int nr);
+
+unsigned int xfs_rui_log_space(unsigned int nr);
+unsigned int xfs_rud_log_space(void);
+
+unsigned int xfs_bui_log_space(unsigned int nr);
+unsigned int xfs_bud_log_space(void);
+
+unsigned int xfs_cui_log_space(unsigned int nr);
+unsigned int xfs_cud_log_space(void);
+
 #endif /* __LIBXFS_DEFER_ITEM_H_ */
libxfs/defer_item.c +51 -0

54 lines not on this page · raw message

Reviewed-by on v1 4/6
#6 lore raw

Re: [PATCH 4/6] libxfs: add helpers to compute log item overhead

Quoted 8 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add selected helpers to estimate the transaction reservation required to write various log intent and buffer items to the log. These helpers will be used by the online repair code for more precise estimations of how much work can be done in a single transaction.

LGTM

Quoted 89 lines
---
 libxfs/defer_item.h |   14 ++++++++++++++
 libxfs/defer_item.c |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
libxfs/defer_item.h +14 -0
index 93cf1eed58a382..325a6f7b2dcbce 100644
@@ -39,4 +39,18 @@ struct xfs_refcount_intent;
 void xfs_refcount_defer_add(struct xfs_trans *tp,
 		struct xfs_refcount_intent *ri);
 
+/* log intent size calculations */
+
+unsigned int xfs_efi_log_space(unsigned int nr);
+unsigned int xfs_efd_log_space(unsigned int nr);
+
+unsigned int xfs_rui_log_space(unsigned int nr);
+unsigned int xfs_rud_log_space(void);
+
+unsigned int xfs_bui_log_space(unsigned int nr);
+unsigned int xfs_bud_log_space(void);
+
+unsigned int xfs_cui_log_space(unsigned int nr);
+unsigned int xfs_cud_log_space(void);
+
 #endif /* __LIBXFS_DEFER_ITEM_H_ */
libxfs/defer_item.c +51 -0
index 6beefa6a439980..4530583ddabae1 100644
@@ -942,3 +942,54 @@ const struct xfs_defer_op_type xfs_exchmaps_defer_type = {
 	.finish_item	= xfs_exchmaps_finish_item,
 	.cancel_item	= xfs_exchmaps_cancel_item,
 };
+
+/* log intent size calculations */
+
+static inline unsigned int
+xlog_item_space(
+	unsigned int	niovecs,
+	unsigned int	nbytes)
+{
+	nbytes += niovecs * (sizeof(uint64_t) + sizeof(struct xlog_op_header));
+	return round_up(nbytes, sizeof(uint64_t));
+}
+
+unsigned int xfs_efi_log_space(unsigned int nr)
+{
+	return xlog_item_space(1, xfs_efi_log_format_sizeof(nr));
+}
+
+unsigned int xfs_efd_log_space(unsigned int nr)
+{
+	return xlog_item_space(1, xfs_efd_log_format_sizeof(nr));
+}
+
+unsigned int xfs_rui_log_space(unsigned int nr)
+{
+	return xlog_item_space(1, xfs_rui_log_format_sizeof(nr));
+}
+
+unsigned int xfs_rud_log_space(void)
+{
+	return xlog_item_space(1, sizeof(struct xfs_rud_log_format));
+}
+
+unsigned int xfs_bui_log_space(unsigned int nr)
+{
+	return xlog_item_space(1, xfs_bui_log_format_sizeof(nr));
+}
+
+unsigned int xfs_bud_log_space(void)
+{
+	return xlog_item_space(1, sizeof(struct xfs_bud_log_format));
+}
+
+unsigned int xfs_cui_log_space(unsigned int nr)
+{
+	return xlog_item_space(1, xfs_cui_log_format_sizeof(nr));
+}
+
+unsigned int xfs_cud_log_space(void)
+{
+	return xlog_item_space(1, sizeof(struct xfs_cud_log_format));
+}
Signature · 1 line
- Andrey
#7 lore raw

PATCH 5/6 xfs: add xfs_calc_atomic_write_unit_max()

In reply to Darrick J. Wong

From: John Garry <[email protected]>

Source kernel commit: 0c438dcc31504bf4f50b20dc52f8f5ca7fab53e2

Now that CoW-based atomic writes are supported, update the max size of an atomic write for the data device.

The limit of a CoW-based atomic write will be the limit of the number of logitems which can fit into a single transaction.

In addition, the max atomic write size needs to be aligned to the agsize. Limit the size of atomic writes to the greatest power-of-two factor of the agsize so that allocations for an atomic write will always be aligned compatibly with the alignment requirements of the storage.

Function xfs_atomic_write_logitems() is added to find the limit the number of log items which can fit in a single transaction.

Amend the max atomic write computation to create a new transaction reservation type, and compute the maximum size of an atomic write completion (in fsblocks) based on this new transaction reservation. Initially, tr_atomic_write is a clone of tr_itruncate, which provides a reasonable level of parallelism. In the next patch, we'll add a mount option so that sysadmins can configure their own limits.

[djwong: use a new reservation type for atomic write ioends, refactor group limit calculations]

---
 include/xfs_trace.h     |    2 +
 libxfs/xfs_trans_resv.h |    2 +
 libxfs/xfs_trans_resv.c |   90 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 94 insertions(+)
include/xfs_trace.h +2 -0
index 30166c11dd597b..965c109cc1941a 100644
@@ -64,6 +64,8 @@
 #define trace_xfs_attr_rmtval_alloc(...)	((void) 0)
 #define trace_xfs_attr_rmtval_remove_return(...) ((void) 0)
 
+#define trace_xfs_calc_max_atomic_write_fsblocks(...) ((void) 0)
+
 #define trace_xfs_log_recover_item_add_cont(a,b,c,d)	((void) 0)
 #define trace_xfs_log_recover_item_add(a,b,c,d)	((void) 0)
 
libxfs/xfs_trans_resv.h +2 -0
index 670045d417a65f..a6d303b836883f 100644
@@ -121,4 +121,6 @@ unsigned int xfs_calc_itruncate_reservation_minlogsize(struct xfs_mount *mp);
 unsigned int xfs_calc_write_reservation_minlogsize(struct xfs_mount *mp);
 unsigned int xfs_calc_qm_dqalloc_reservation_minlogsize(struct xfs_mount *mp);
 
+xfs_extlen_t xfs_calc_max_atomic_write_fsblocks(struct xfs_mount *mp);
+
 #endif	/* __XFS_TRANS_RESV_H__ */
libxfs/xfs_trans_resv.c +90 -0

99 lines not on this page · raw message

#8 lore raw

PATCH 6/6 xfs: allow sysadmins to specify a maximum atomic write limit at mount time

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Source kernel commit: 4528b9052731f14c1a9be16b98e33c9401e6d1bc

Introduce a mount option to allow sysadmins to specify the maximum size of an atomic write. If the filesystem can work with the supplied value, that becomes the new guaranteed maximum.

The value mustn't be too big for the existing filesystem geometry (max write size, max AG/rtgroup size). We dynamically recompute the tr_atomic_write transaction reservation based on the given block size, check that the current log size isn't less than the new minimum log size constraints, and set a new maximum.

The actual software atomic write max is still computed based off of tr_atomic_ioend the same way it has for the past few commits. Note also that xfs_calc_atomic_write_log_geometry is non-static because mkfs will need that.

---
 include/platform_defs.h |   14 ++++++++++
 include/xfs_trace.h     |    1 +
 libxfs/xfs_trans_resv.h |    4 +++
 libxfs/xfs_trans_resv.c |   69 +++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 88 insertions(+)
include/platform_defs.h +14 -0
index 9af7b4318f8917..74a00583ebd6cb 100644
@@ -261,6 +261,20 @@ static inline bool __must_check __must_check_overflow(bool overflow)
 	__builtin_add_overflow(__a, __b, __d);	\
 }))
 
+/**
+ * check_mul_overflow() - Calculate multiplication with overflow checking
+ * @a: first factor
+ * @b: second factor
+ * @d: pointer to store product
+ *
+ * Returns true on wrap-around, false otherwise.
+ *
+ * *@d holds the results of the attempted multiplication, regardless of whether
+ * wrap-around occurred.
+ */
+#define check_mul_overflow(a, b, d)	\
+	__must_check_overflow(__builtin_mul_overflow(a, b, d))
+
 /**
  * abs_diff - return absolute value of the difference between the arguments
  * @a: the first argument
include/xfs_trace.h +1 -0
index 965c109cc1941a..6bbf4007cbbbd9 100644
@@ -65,6 +65,7 @@
 #define trace_xfs_attr_rmtval_remove_return(...) ((void) 0)
 
 #define trace_xfs_calc_max_atomic_write_fsblocks(...) ((void) 0)
+#define trace_xfs_calc_max_atomic_write_log_geometry(...) ((void) 0)
 
 #define trace_xfs_log_recover_item_add_cont(a,b,c,d)	((void) 0)
 #define trace_xfs_log_recover_item_add(a,b,c,d)	((void) 0)
libxfs/xfs_trans_resv.h +4 -0
index a6d303b836883f..336279e0fc6137 100644
@@ -122,5 +122,9 @@ unsigned int xfs_calc_write_reservation_minlogsize(struct xfs_mount *mp);
 unsigned int xfs_calc_qm_dqalloc_reservation_minlogsize(struct xfs_mount *mp);
 
 xfs_extlen_t xfs_calc_max_atomic_write_fsblocks(struct xfs_mount *mp);
+xfs_extlen_t xfs_calc_atomic_write_log_geometry(struct xfs_mount *mp,
+		xfs_extlen_t blockcount, unsigned int *new_logres);
+int xfs_calc_atomic_write_reservation(struct xfs_mount *mp,
+		xfs_extlen_t blockcount);
 
 #endif	/* __XFS_TRANS_RESV_H__ */
libxfs/xfs_trans_resv.c +69 -0

72 lines not on this page · raw message

#9 lore raw

PATCH 2/3 xfsprogs: atomic writes

In reply to Darrick J. Wong

Hi all,

Utility updates for dumping atomic writes configuration and formatting filesystems to take advantage of the new functionality.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

Comments and questions are, as always, welcome.

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=atomic-writes-6.16

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=atomic-writes-6.16
---
Commits in this patchset:
* libfrog: move statx.h from io/ to libfrog/
* xfs_db: create an untorn_max subcommand
* xfs_io: dump new atomic_write_unit_max_opt statx field
* mkfs: don't complain about overly large auto-detected log stripe units
* mkfs: autodetect log stripe unit for external log devices
* mkfs: try to align AG size based on atomic write capabilities
* mkfs: allow users to configure the desired maximum atomic write size

---
 include/bitops.h         |   12 ++
 include/libxfs.h         |    1 
 libfrog/statx.h          |   23 ++++
 libxfs/libxfs_api_defs.h |    5 +
 libxfs/topology.h        |    6 +
 db/logformat.c           |  129 ++++++++++++++++++++++++
 io/stat.c                |   21 +---
 libfrog/Makefile         |    1 
 libxfs/topology.c        |   36 +++++++
 m4/package_libcdev.m4    |    2 
 man/man8/mkfs.xfs.8.in   |    7 +
 man/man8/xfs_db.8        |   10 ++
 mkfs/xfs_mkfs.c          |  251 +++++++++++++++++++++++++++++++++++++++++++++-
 13 files changed, 474 insertions(+), 30 deletions(-)
 rename io/statx.h => libfrog/statx.h (94%)
#10 lore raw

PATCH 1/7 libfrog: move statx.h from io/ to libfrog/

From: Darrick J. Wong <[email protected]>

Move this header file so we can use the declaration and wrappers in other parts of xfsprogs.

---
 libfrog/statx.h  |   17 +++++++++++++++++
 io/stat.c        |   20 ++------------------
 libfrog/Makefile |    1 +
 3 files changed, 20 insertions(+), 18 deletions(-)
 rename io/statx.h => libfrog/statx.h (96%)
libfrog/statx.h renamed from io/statx.h +17 -0
similarity index 96%
rename from io/statx.h
rename to libfrog/statx.h
index f7ef1d2784a2a9..b76dfae21e7092 100644
@@ -146,7 +146,24 @@ struct statx {
 	__u64	__spare3[9];	/* Spare space for future expansion */
 	/* 0x100 */
 };
+
+static inline ssize_t
+statx(
+	int		dfd,
+	const char	*filename,
+	unsigned int	flags,
+	unsigned int	mask,
+	struct statx	*buffer)
+{
+#ifdef __NR_statx
+	return syscall(__NR_statx, dfd, filename, flags, mask, buffer);
+#else
+	errno = ENOSYS;
+	return -1;
 #endif
+}
+
+#endif /* OVERRIDE_SYSTEM_STATX */
 
 #ifndef STATX_TYPE
 /*
io/stat.c +2 -18
index c3a4bb15229ee5..46475df343470c 100644
@@ -14,7 +14,7 @@
 #include "input.h"
 #include "init.h"
 #include "io.h"
-#include "statx.h"
+#include "libfrog/statx.h"
 #include "libxfs.h"
 #include "libfrog/logging.h"
 #include "libfrog/fsgeom.h"
@@ -305,22 +305,6 @@ statfs_f(
 	return 0;
 }
 
-static ssize_t
-_statx(
-	int		dfd,
-	const char	*filename,
-	unsigned int	flags,
-	unsigned int	mask,
-	struct statx	*buffer)
-{
-#ifdef __NR_statx
-	return syscall(__NR_statx, dfd, filename, flags, mask, buffer);
-#else
-	errno = ENOSYS;
-	return -1;
-#endif
-}
-
 struct statx_masks {
 	const char	*name;
 	unsigned int	mask;
@@ -525,7 +509,7 @@ statx_f(
 		return command_usage(&statx_cmd);
 
 	memset(&stx, 0xbf, sizeof(stx));
-	if (_statx(file->fd, "", atflag | AT_EMPTY_PATH, mask, &stx) < 0) {
+	if (statx(file->fd, "", atflag | AT_EMPTY_PATH, mask, &stx) < 0) {
 		perror("statx");
 		exitcode = 1;
 		return 0;
libfrog/Makefile +1 -0
index b64ca4597f4ea9..560bad417ee434 100644
@@ -62,6 +62,7 @@ ptvar.h \
 radix-tree.h \
 randbytes.h \
 scrub.h \
+statx.h \
 workqueue.h
 
 GETTEXT_PY = \
#11 lore raw

PATCH 2/7 xfs_db: create an untorn_max subcommand

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a debugger command to compute the either the logres needed to perform an untorn cow write completion for a given number of blocks; or the number of blocks that can be completed given a log reservation.

---
 include/libxfs.h         |    1 
 libxfs/libxfs_api_defs.h |    4 +
 db/logformat.c           |  129 ++++++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_db.8        |   10 ++++
 4 files changed, 144 insertions(+)
include/libxfs.h +1 -0
index b968a2b88da372..1e0d1a48fbb698 100644
@@ -102,6 +102,7 @@ struct iomap;
 #include "xfs_rtbitmap.h"
 #include "xfs_rtrmap_btree.h"
 #include "xfs_ag_resv.h"
+#include "defer_item.h"
 
 #ifndef ARRAY_SIZE
 #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
libxfs/libxfs_api_defs.h +4 -0
index c5fcb5e3229ae4..4bd02c57b496e6 100644
@@ -108,6 +108,10 @@
 #define xfs_bunmapi			libxfs_bunmapi
 #define xfs_bwrite			libxfs_bwrite
 #define xfs_calc_dquots_per_chunk	libxfs_calc_dquots_per_chunk
+#define xfs_calc_finish_bui_reservation	libxfs_calc_finish_bui_reservation
+#define xfs_calc_finish_cui_reservation	libxfs_calc_finish_cui_reservation
+#define xfs_calc_finish_efi_reservation	libxfs_calc_finish_efi_reservation
+#define xfs_calc_finish_rui_reservation	libxfs_calc_finish_rui_reservation
 #define xfs_cntbt_init_cursor		libxfs_cntbt_init_cursor
 #define xfs_compute_rextslog		libxfs_compute_rextslog
 #define xfs_compute_rgblklog		libxfs_compute_rgblklog
db/logformat.c +129 -0

137 lines not on this page · raw message

man/man8/xfs_db.8 +10 -0
index 2a9322560584b0..1e85aebbb5b27c 100644
@@ -1366,6 +1366,16 @@ .SH COMMANDS
 .IR name .
 The file being targetted will not be put on the iunlink list.
 .TP
+.BI "untorn_write_max [\-b " blockcount "|\-l " logres "]"
+If
+.B -l
+is specified, compute the maximum (in fsblocks) untorn write that we can
+emulate with copy on write given a log reservation size (in bytes).
+If
+.B -b
+is specified, compute the log reservation size that would be needed to
+emulate an untorn write of the given number of fsblocks.
+.TP
 .BI "uuid [" uuid " | " generate " | " rewrite " | " restore ]
 Set the filesystem universally unique identifier (UUID).
 The filesystem UUID can be used by
#12 lore raw

PATCH 3/7 xfs_io: dump new atomic_write_unit_max_opt statx field

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Dump the new atomic writes statx field that's being submitted for 6.16.

---
 libfrog/statx.h       |    6 +++++-
 io/stat.c             |    1 +
 m4/package_libcdev.m4 |    2 +-
 3 files changed, 7 insertions(+), 2 deletions(-)
libfrog/statx.h +5 -1
index b76dfae21e7092..e11e2d8f49fa5f 100644
@@ -143,7 +143,11 @@ struct statx {
 	__u32	stx_dio_read_offset_align;
 
 	/* 0xb8 */
-	__u64	__spare3[9];	/* Spare space for future expansion */
+	/* Optimised max atomic write unit in bytes */
+	__u32	stx_atomic_write_unit_max_opt;
+	__u32	__spare2[1];
+	/* 0xc0 */
+	__u64	__spare3[8];	/* Spare space for future expansion */
 	/* 0x100 */
 };
 
io/stat.c +1 -0
index 46475df343470c..c257037aa8eec3 100644
@@ -396,6 +396,7 @@ dump_raw_statx(struct statx *stx)
 	printf("stat.atomic_write_unit_max = %u\n", stx->stx_atomic_write_unit_max);
 	printf("stat.atomic_write_segments_max = %u\n", stx->stx_atomic_write_segments_max);
 	printf("stat.dio_read_offset_align = %u\n", stx->stx_dio_read_offset_align);
+	printf("stat.atomic_write_unit_max_opt = %u\n", stx->stx_atomic_write_unit_max_opt);
 	return 0;
 }
 
m4/package_libcdev.m4 +1 -1
index 61353d0aa9d536..b77ac1a7580a80 100644
@@ -126,7 +126,7 @@ AC_DEFUN([AC_NEED_INTERNAL_FSCRYPT_POLICY_V2],
 AC_DEFUN([AC_NEED_INTERNAL_STATX],
   [ AC_CHECK_TYPE(struct statx,
       [
-        AC_CHECK_MEMBER(struct statx.stx_dio_read_offset_align,
+        AC_CHECK_MEMBER(struct statx.stx_atomic_write_unit_max_opt,
           ,
           need_internal_statx=yes,
           [#include <linux/stat.h>]
#13 lore raw

PATCH 4/7 mkfs: don't complain about overly large auto-detected log stripe units

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If mkfs declines to apply what it thinks is an overly large data device stripe unit to the log device, it should only log a message about that if the lsunit parameter was actually supplied by the caller. It should not do that when the lsunit was autodetected from the block devices.

The cli parameters are zero-initialized in main and always have been.

Cc <[email protected]> # v4.15.0
Fixes 2f44b1b0e5adc4 ("mkfs: rework stripe calculations")
---
 mkfs/xfs_mkfs.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mkfs/xfs_mkfs.c +1 -1
index 812241c49a5494..8b946f3ef817da 100644
@@ -3629,7 +3629,7 @@ _("log stripe unit (%d) must be a multiple of the block size (%d)\n"),
 	if (cfg->sb_feat.log_version == 2 &&
 	    cfg->lsunit * cfg->blocksize > 256 * 1024) {
 		/* Warn only if specified on commandline */
-		if (cli->lsu || cli->lsunit != -1) {
+		if (cli->lsu || cli->lsunit) {
 			fprintf(stderr,
 _("log stripe unit (%d bytes) is too large (maximum is 256KiB)\n"
   "log stripe unit adjusted to 32KiB\n"),
#14 lore raw

PATCH 5/7 mkfs: autodetect log stripe unit for external log devices

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If we're using an external log device and the caller doesn't give us a lsunit, use the block device geometry (if present) to set it.

---
 mkfs/xfs_mkfs.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
mkfs/xfs_mkfs.c +6 -1
index 8b946f3ef817da..b6de13cebc93ed 100644
@@ -3615,7 +3615,8 @@ _("log stripe unit (%d) must be a multiple of the block size (%d)\n"),
 	}
 
 	/*
-	 * check that log sunit is modulo fsblksize or default it to dsunit.
+	 * check that log sunit is modulo fsblksize; default it to dsunit for
+	 * an internal log; or the log device stripe unit if it's external.
 	 */
 	if (lsunit) {
 		/* convert from 512 byte blocks to fs blocks */
@@ -3624,6 +3625,10 @@ _("log stripe unit (%d) must be a multiple of the block size (%d)\n"),
 		   cfg->loginternal && cfg->dsunit) {
 		/* lsunit and dsunit now in fs blocks */
 		cfg->lsunit = cfg->dsunit;
+	} else if (cfg->sb_feat.log_version == 2 &&
+		   !cfg->loginternal) {
+		/* use the external log device properties */
+		cfg->lsunit = DTOBT(ft->log.sunit, cfg->blocklog);
 	}
 
 	if (cfg->sb_feat.log_version == 2 &&
#15 lore raw

PATCH 6/7 mkfs: try to align AG size based on atomic write capabilities

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Try to align the AG size to the maximum hardware atomic write unit so that we can give users maximum flexibility in choosing an RWF_ATOMIC write size.

---
 libxfs/topology.h |    6 ++++--
 libxfs/topology.c |   36 ++++++++++++++++++++++++++++++++++++
 mkfs/xfs_mkfs.c   |   48 +++++++++++++++++++++++++++++++++++++++++++-----
 3 files changed, 83 insertions(+), 7 deletions(-)
libxfs/topology.h +4 -2
index 207a8a7f150556..f0ca65f3576e92 100644
@@ -13,8 +13,10 @@
 struct device_topology {
 	int	logical_sector_size;	/* logical sector size */
 	int	physical_sector_size;	/* physical sector size */
-	int	sunit;		/* stripe unit */
-	int	swidth;		/* stripe width  */
+	int	sunit;			/* stripe unit */
+	int	swidth;			/* stripe width  */
+	int	awu_min;		/* min atomic write unit in bbcounts */
+	int	awu_max;		/* max atomic write unit in bbcounts */
 };
 
 struct fs_topology {
libxfs/topology.c +36 -0
index 96ee74b61b30f5..7764687beac000 100644
@@ -4,11 +4,18 @@
  * All Rights Reserved.
  */
 
+#ifdef OVERRIDE_SYSTEM_STATX
+#define statx sys_statx
+#endif
+#include <fcntl.h>
+#include <sys/stat.h>
+
 #include "libxfs_priv.h"
 #include "libxcmd.h"
 #include <blkid/blkid.h>
 #include "xfs_multidisk.h"
 #include "libfrog/platform.h"
+#include "libfrog/statx.h"
 
 #define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
 #define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
@@ -278,6 +285,34 @@ blkid_get_topology(
 		device);
 }
 
+static void
+get_hw_atomic_writes_topology(
+	struct libxfs_dev	*dev,
+	struct device_topology	*dt)
+{
+	struct statx		sx;
+	int			fd;
+	int			ret;
+
+	fd = open(dev->name, O_RDONLY);
+	if (fd < 0)
+		return;
+
+	ret = statx(fd, "", AT_EMPTY_PATH, STATX_WRITE_ATOMIC, &sx);
+	if (ret)
+		goto out_close;
+
+	if (!(sx.stx_mask & STATX_WRITE_ATOMIC))
+		goto out_close;
+
+	dt->awu_min = sx.stx_atomic_write_unit_min >> 9;
+	dt->awu_max = max(sx.stx_atomic_write_unit_max_opt,
+			  sx.stx_atomic_write_unit_max) >> 9;
+
+out_close:
+	close(fd);
+}
+
 static void
 get_device_topology(
 	struct libxfs_dev	*dev,
@@ -316,6 +351,7 @@ get_device_topology(
 		}
 	} else {
 		blkid_get_topology(dev->name, dt, force_overwrite);
+		get_hw_atomic_writes_topology(dev, dt);
 	}
 
 	ASSERT(dt->logical_sector_size);
mkfs/xfs_mkfs.c +43 -5

79 lines not on this page · raw message

#16 lore raw

PATCH 7/7 mkfs: allow users to configure the desired maximum atomic write size

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Allow callers of mkfs.xfs to specify a desired maximum atomic write size. This value will cause the log size to be adjusted to support software atomic writes, and the AG size to be aligned to support hardware atomic writes.

---
 include/bitops.h         |   12 +++
 libxfs/libxfs_api_defs.h |    1 
 man/man8/mkfs.xfs.8.in   |    7 ++
 mkfs/xfs_mkfs.c          |  194 ++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 213 insertions(+), 1 deletion(-)
include/bitops.h +12 -0
index 1f1adceccf5d2b..d0c55827044e54 100644
@@ -113,4 +113,16 @@ static inline int lowbit64(uint64_t v)
 	return n - 1;
 }
 
+/**
+ * __rounddown_pow_of_two() - round down to nearest power of two
+ * @n: value to round down
+ */
+static inline __attribute__((const))
+unsigned long __rounddown_pow_of_two(unsigned long n)
+{
+	return 1UL << (fls_long(n) - 1);
+}
+
+#define rounddown_pow_of_two(n) __rounddown_pow_of_two(n)
+
 #endif
libxfs/libxfs_api_defs.h +1 -0
index 4bd02c57b496e6..fe00e19bada9d8 100644
@@ -107,6 +107,7 @@
 #define xfs_buftarg_drain		libxfs_buftarg_drain
 #define xfs_bunmapi			libxfs_bunmapi
 #define xfs_bwrite			libxfs_bwrite
+#define xfs_calc_atomic_write_log_geometry	libxfs_calc_atomic_write_log_geometry
 #define xfs_calc_dquots_per_chunk	libxfs_calc_dquots_per_chunk
 #define xfs_calc_finish_bui_reservation	libxfs_calc_finish_bui_reservation
 #define xfs_calc_finish_cui_reservation	libxfs_calc_finish_cui_reservation
man/man8/mkfs.xfs.8.in +7 -0
index bc80493187f6f9..5f59d4b2da6e02 100644
@@ -742,6 +742,13 @@ .SH OPTIONS
 directories, symbolic links, and realtime metadata files.
 This feature is disabled by default.
 This feature is only available for filesystems formatted with -m crc=1.
+.TP
+.BI max_atomic_write[= value]
+When enabled, application programs can use the RWF_ATOMIC write flag to
+persist changes of up to this size without tearing.
+The default is chosen to allow a reasonable amount of scalability.
+This value must also be passed via mount option.
+This feature is only available for filesystems formatted with reflink.
 .RE
 .PP
 .PD 0
mkfs/xfs_mkfs.c +193 -1

266 lines not on this page · raw message

#17 lore raw

PATCH 3/3 xfs_scrub: drop EXPERIMENTAL warning

In reply to Darrick J. Wong

Hi all,

Drop the EXPERIMENTAL warning on xfs_scrub now that we've done so for the kernel.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

Comments and questions are, as always, welcome.

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-6.16
---
Commits in this patchset:
* xfs_scrub: remove EXPERIMENTAL warnings

---
 man/man8/xfs_scrub.8 |    6 ------
 scrub/xfs_scrub.c    |    3 ---
 2 files changed, 9 deletions(-)
#18 lore raw

PATCH 1/1 xfs_scrub: remove EXPERIMENTAL warnings

From: Darrick J. Wong <[email protected]>

The kernel code for online fsck has been stable for a year, and there haven't been any major changes to the program in quite some time, so let's drop the experimental warnings.

---
 man/man8/xfs_scrub.8 |    6 ------
 scrub/xfs_scrub.c    |    3 ---
 2 files changed, 9 deletions(-)
man/man8/xfs_scrub.8 +0 -6
index 1ed4b176b6a35a..8b2a990831a2cc 100644
@@ -13,12 +13,6 @@ .SH DESCRIPTION
 .B xfs_scrub
 attempts to check and repair all metadata in a mounted XFS filesystem.
 .PP
-.B WARNING!
-This program is
-.BR EXPERIMENTAL ","
-which means that its behavior and interface
-could change at any time!
-.PP
 .B xfs_scrub
 asks the kernel to scrub all metadata objects in the filesystem.
 Metadata records are scanned for obviously bad values and then
scrub/xfs_scrub.c +0 -3
index 90897cc26cd71d..3dba972a7e8d2a 100644
@@ -730,9 +730,6 @@ main(
 	hist_init(&ctx.datadev_hist);
 	hist_init(&ctx.rtdev_hist);
 
-	fprintf(stdout, "EXPERIMENTAL xfs_scrub program in use! Use at your own risk!\n");
-	fflush(stdout);
-
 	progname = basename(argv[0]);
 	setlocale(LC_ALL, "");
 	bindtextdomain(PACKAGE, LOCALEDIR);