Skip to contentSkip to conversation

xfsprogs: catch us up to 6.13

PATCHBOMB New Darrick J. Wong 1 participant 14 messages Jan 16, 2025

lore
Share & tools
lore
#0 lore raw

Hi Andrey,

This patchbomb is a roll-up of all the stuff I still had to send for the xfsprogs 6.13 release. Of the nine patches, the first one is a libxfs sync for something that went in midway through the 6.13 cycle, and the last one adds a piece of missing functionality that fell out during the rest of the 6.13 xfsprogs review. Patches 2-7 are bug fixes.

The entire pile have been reviewed, so there are pull requests at the end if you'd like them.

--D

#1 lore raw

PATCH 1/2 xfsprogs: new libxfs code from kernel 6.13

Hi all,

Port kernel libxfs code to userspace.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

Comments and questions are, as always, welcome.

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=libxfs-sync-6.13
---
Commits in this patchset:
* xfs: don't return an error from xfs_update_last_rtgroup_size for !XFS_RT

---
 libxfs/xfs_rtgroup.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
#2 lore raw

PATCH 1/1 xfs: don't return an error from xfs_update_last_rtgroup_size for !XFS_RT

From: Christoph Hellwig <[email protected]>

Source kernel commit: 7ee7c9b39ed36caf983706f5b893cc5c37a79071

Non-rtg file systems have a fake RT group even if they do not have a RT device, and thus an rgcount of 1. Ensure xfs_update_last_rtgroup_size doesn't fail when called for !XFS_RT to handle this case.

Fixes 87fe4c34a383 ("xfs: create incore realtime group structures")
---
 libxfs/xfs_rtgroup.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/xfs_rtgroup.h +1 -1
index 7e7e491ff06fa5..2d7822644efff0 100644
@@ -272,7 +272,7 @@ static inline int xfs_initialize_rtgroups(struct xfs_mount *mp,
 }
 
 # define xfs_rtgroup_extents(mp, rgno)		(0)
-# define xfs_update_last_rtgroup_size(mp, rgno)	(-EOPNOTSUPP)
+# define xfs_update_last_rtgroup_size(mp, rgno)	(0)
 # define xfs_rtgroup_lock(rtg, gf)		((void)0)
 # define xfs_rtgroup_unlock(rtg, gf)		((void)0)
 # define xfs_rtgroup_trans_join(tp, rtg, gf)	((void)0)
#3 lore raw

PATCH 2/2 xfs: bug fixes for 6.13

In reply to Darrick J. Wong

Hi all,

Bug fixes for 6.13.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

Comments and questions are, as always, welcome.

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=xfs-fixes-6.13

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=xfs-fixes-6.13
---
Commits in this patchset:
* xfs_db: fix multiple dblock commands
* xfs_repair: don't obliterate return codes
* libxfs: fix uninit variable in libxfs_alloc_file_space
* xfs_db: improve error message when unknown btree type given to btheight
* mkfs: fix parsing of value-less -d/-l concurrency cli option
* m4: fix statx override selection if /usr/include doesn't define it
* build: initialize stack variables to zero by default
* mkfs: allow sizing realtime allocation groups for concurrency

---
 configure.ac            |    1 
 db/block.c              |    4 +
 db/btheight.c           |    6 ++
 include/builddefs.in    |    2 -
 libxfs/util.c           |    2 -
 m4/package_libcdev.m4   |    2 -
 m4/package_sanitizer.m4 |   14 +++++
 man/man8/mkfs.xfs.8.in  |   28 +++++++++
 mkfs/xfs_mkfs.c         |  144 +++++++++++++++++++++++++++++++++++++++++++++--
 repair/quotacheck.c     |    2 -
 10 files changed, 195 insertions(+), 10 deletions(-)
#4 lore raw

PATCH 1/8 xfs_db: fix multiple dblock commands

From: Darrick J. Wong <[email protected]>

Tom Samstag reported that running the following sequence of commands no longer works quite right:

Quoted 8 lines

inode [inodenum]
dblock 0
p
dblock 1
p
dblock 2
p
[etc]

Mr. Samstag looked into the source code and discovered that the dblock_f is incorrectly accessing iocur_top->data outside of the push_cur -> set_cur_inode -> pop_cur sequence that this function uses to compute the type of the file data. In other words, it's using whatever's on top of the stack at the start of the function. For the "dblock 0" case above this is the inode, but for the "dblock 1" case this is the contents of file data block 0, not an inode.

Fix this by relocating the check to the correct place.

Reported-by [email protected]
Cc <[email protected]> # v6.12.0
Fixes b05a31722f5d4c ("xfs_db: access realtime file blocks")
---
 db/block.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
db/block.c +3 -1
index 00830a3d57e1df..2f1978c41f3094 100644
@@ -246,6 +246,7 @@ dblock_f(
 	int		nb;
 	xfs_extnum_t	nex;
 	char		*p;
+	bool		isrt;
 	typnm_t		type;
 
 	bno = (xfs_fileoff_t)strtoull(argv[1], &p, 0);
@@ -255,6 +256,7 @@ dblock_f(
 	}
 	push_cur();
 	set_cur_inode(iocur_top->ino);
+	isrt = is_rtfile(iocur_top->data);
 	type = inode_next_type();
 	pop_cur();
 	if (type == TYP_NONE) {
@@ -273,7 +275,7 @@ dblock_f(
 	ASSERT(typtab[type].typnm == type);
 	if (nex > 1)
 		make_bbmap(&bbmap, nex, bmp);
-	if (is_rtfile(iocur_top->data))
+	if (isrt)
 		set_rt_cur(&typtab[type], xfs_rtb_to_daddr(mp, dfsbno),
 				nb * blkbb, DB_RING_ADD,
 				nex > 1 ? &bbmap : NULL);
#5 lore raw

PATCH 2/8 xfs_repair: don't obliterate return codes

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Don't clobber error here, it's err2 that's the temporary variable.

Coverity-id: 1637363

Fixes b790ab2a303d58 ("xfs_repair: support quota inodes in the metadata directory")
---
 repair/quotacheck.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
repair/quotacheck.c +1 -1
index 8c7339b267d8e6..7953144c3f416b 100644
@@ -691,7 +691,7 @@ discover_quota_inodes(
 	err2 = mark_quota_inode(tp, dp, XFS_DQTYPE_GROUP);
 	if (err2 && !error)
 		error = err2;
-	error = mark_quota_inode(tp, dp, XFS_DQTYPE_PROJ);
+	err2 = mark_quota_inode(tp, dp, XFS_DQTYPE_PROJ);
 	if (err2 && !error)
 		error = err2;
 
#6 lore raw

PATCH 3/8 libxfs: fix uninit variable in libxfs_alloc_file_space

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix this uninitialized variable.

Coverity-id: 1637359

Fixes b48164b8cd7618 ("libxfs: resync libxfs_alloc_file_space interface with the kernel")
---
 libxfs/util.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/util.c +1 -1
index 4a9dd254083a63..3597850ddccb9a 100644
@@ -193,7 +193,7 @@ libxfs_alloc_file_space(
 	int			rt;
 	xfs_trans_t		*tp;
 	xfs_bmbt_irec_t		imaps[1], *imapp;
-	int			error;
+	int			error = 0;
 
 	if (len <= 0)
 		return -EINVAL;
#7 lore raw

PATCH 4/8 xfs_db: improve error message when unknown btree type given to btheight

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

I found accidentally that if you do this (note 'rmap', not 'rmapbt'):

xfs_db /dev/sda -c 'btheight -n 100 rmap'

The program spits back "Numerical result out of range". That's the result of it failing to match "rmap" against a known btree type, and falling back to parsing the string as if it were a btree geometry description.

Improve this a little by checking that there's at least one semicolon in the string so that the error message improves to:

"rmap: expected a btree geometry specification"

Fixes cb1e69c564c1e0 ("xfs_db: add a function to compute btree geometry")
---
 db/btheight.c |    6 ++++++
 1 file changed, 6 insertions(+)
db/btheight.c +6 -0
index 6643489c82c4c9..98165b522e4f6f 100644
@@ -145,6 +145,12 @@ construct_records_per_block(
 		}
 	}
 
+	p = strchr(tag, ':');
+	if (!p) {
+		fprintf(stderr, _("%s: expected a btree geometry specification.\n"), tag);
+		return -1;
+	}
+
 	toktag = strdup(tag);
 	ret = -1;
 
#8 lore raw

PATCH 5/8 mkfs: fix parsing of value-less -d/-l concurrency cli option

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

It's supposed to be possible to specify the -d concurrency option with no value in order to get mkfs calculate the agcount from the number of CPUs. Unfortunately I forgot to handle that case (optarg is null) so mkfs crashes instead. Fix that.

Fixes 9338bc8b1bf073 ("mkfs: allow sizing allocation groups for concurrency")
---
 mkfs/xfs_mkfs.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mkfs/xfs_mkfs.c +2 -2
index 956cc295489342..deaac2044b94dd 100644
@@ -1722,7 +1722,7 @@ set_data_concurrency(
 	 * "nr_cpus" or "1" means set the concurrency level to the CPU count.
 	 * If this cannot be determined, fall back to the default AG geometry.
 	 */
-	if (!strcmp(value, "nr_cpus"))
+	if (!value || !strcmp(value, "nr_cpus"))
 		optnum = 1;
 	else
 		optnum = getnum(value, opts, subopt);
@@ -1867,7 +1867,7 @@ set_log_concurrency(
 	 * "nr_cpus" or 1 means set the concurrency level to the CPU count.  If
 	 * this cannot be determined, fall back to the default computation.
 	 */
-	if (!strcmp(value, "nr_cpus"))
+	if (!value || !strcmp(value, "nr_cpus"))
 		optnum = 1;
 	else
 		optnum = getnum(value, opts, subopt);
#9 lore raw

PATCH 6/8 m4: fix statx override selection if /usr/include doesn't define it

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the system headers (aka the ones in /usr/include) do not define struct statx at all, we need to use our internal override. The m4 code doesn't handle this admittedly corner case, but let's fix it for anyone trying to build new xfsprogs on a decade-old distribution.

Fixes 409477af604f46 ("xfs_io: add support for atomic write statx fields")
---
 m4/package_libcdev.m4 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
m4/package_libcdev.m4 +1 -1
index 6db1177350b643..4ef7e8f67a3ba6 100644
@@ -112,7 +112,7 @@ AC_DEFUN([AC_NEED_INTERNAL_STATX],
           need_internal_statx=yes,
           [#include <linux/stat.h>]
         )
-      ],,
+      ],need_internal_statx=yes,
       [#include <linux/stat.h>]
     )
     AC_SUBST(need_internal_statx)
#10 lore raw

PATCH 7/8 build: initialize stack variables to zero by default

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Newer versions of gcc and clang can include the ability to zero stack variables by default. Let's enable it so that we (a) reduce the risk of writing stack contents to disk somewhere and (b) try to reduce unpredictable program behavior based on random stack contents. The kernel added this 6 years ago, so I think it's mature enough for xfsprogs.

---
 configure.ac            |    1 +
 include/builddefs.in    |    2 +-
 m4/package_sanitizer.m4 |   14 ++++++++++++++
 3 files changed, 16 insertions(+), 1 deletion(-)
configure.ac +1 -0
index 224d1d3930bf2f..90ef7925a925d0 100644
@@ -177,6 +177,7 @@ AC_CONFIG_SYSTEMD_SYSTEM_UNIT_DIR
 AC_CONFIG_CROND_DIR
 AC_CONFIG_UDEV_RULE_DIR
 AC_HAVE_BLKID_TOPO
+AC_HAVE_TRIVIAL_AUTO_VAR_INIT
 
 if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then
         AC_PACKAGE_CHECK_UBSAN
include/builddefs.in +1 -1
index ac43b6412c8cbb..82840ec7fd3adb 100644
@@ -146,7 +146,7 @@ ifeq ($(HAVE_LIBURCU_ATOMIC64),yes)
 PCFLAGS += -DHAVE_LIBURCU_ATOMIC64
 endif
 
-SANITIZER_CFLAGS += @addrsan_cflags@ @threadsan_cflags@ @ubsan_cflags@
+SANITIZER_CFLAGS += @addrsan_cflags@ @threadsan_cflags@ @ubsan_cflags@ @autovar_init_cflags@
 SANITIZER_LDFLAGS += @addrsan_ldflags@ @threadsan_ldflags@ @ubsan_ldflags@
 
 # Use special ar/ranlib wrappers if we have lto
m4/package_sanitizer.m4 +14 -0
index 41b729906a27ba..6488f7ebce2f50 100644
@@ -57,3 +57,17 @@ AC_DEFUN([AC_PACKAGE_CHECK_THREADSAN],
     AC_SUBST(threadsan_cflags)
     AC_SUBST(threadsan_ldflags)
   ])
+
+# Check if we have -ftrivial-auto-var-init=zero
+AC_DEFUN([AC_HAVE_TRIVIAL_AUTO_VAR_INIT],
+  [ AC_MSG_CHECKING([if C compiler supports zeroing automatic vars])
+    OLD_CFLAGS="$CFLAGS"
+    TEST_CFLAGS="-ftrivial-auto-var-init=zero"
+    CFLAGS="$CFLAGS $TEST_CFLAGS"
+    AC_LINK_IFELSE([AC_LANG_PROGRAM([])],
+        [AC_MSG_RESULT([yes])]
+        [autovar_init_cflags=$TEST_CFLAGS],
+        [AC_MSG_RESULT([no])])
+    CFLAGS="${OLD_CFLAGS}"
+    AC_SUBST(autovar_init_cflags)
+  ])
#11 lore raw

PATCH 8/8 mkfs: allow sizing realtime allocation groups for concurrency

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a -r concurrency= option to mkfs so that sysadmins can configure the filesystem so that there are enough rtgroups that the specified number of threads can (in theory) can find an uncontended rtgroup from which to allocate space. This has the exact same purpose as the -d concurrency switch that was added for the data device.

---
 man/man8/mkfs.xfs.8.in |   28 ++++++++++
 mkfs/xfs_mkfs.c        |  140 +++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 165 insertions(+), 3 deletions(-)
man/man8/mkfs.xfs.8.in +28 -0
index 32361cf973fcf8..37e3a88e7ac777 100644
@@ -1220,6 +1220,34 @@ .SH OPTIONS
 and
 .B rgsize
 suboptions are mutually exclusive.
+.TP
+.BI concurrency= value
+Create enough realtime allocation groups to handle the desired level of
+concurrency.
+The goal of this calculation scheme is to set the number of rtgroups to an
+integer multiple of the number of writer threads desired, to minimize
+contention of rtgroup locks.
+This scheme will neither create fewer rtgroups than would be created by the
+default configuration, nor will it create rtgroups smaller than 4GB.
+This option is not compatible with the
+.B rgcount
+or
+.B rgsize
+options.
+The magic value
+.I nr_cpus
+or
+.I 1
+or no value at all will set this parameter to the number of active processors
+in the system.
+If the kernel advertises that the realtime device is a non-mechanical storage
+device,
+.B mkfs.xfs
+will use this new geometry calculation scheme.
+The magic value of
+.I 0
+forces use of the older rtgroups geometry calculations that is used for
+mechanical storage.
 .RE
 .PP
 .PD 0
mkfs/xfs_mkfs.c +137 -3

221 lines not on this page · raw message

#12 lore raw

[GIT PULL 1/2] xfsprogs: new libxfs code from kernel 6.13

In reply to Darrick J. Wong

Hi Andrey,

Please pull this branch with changes for xfsprogs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.

The following changes since commit 19bca351dcdfef4a63ede08bcc9f7bdeec10c453:

xfs_io: add extsize command support (2024-12-25 22:14:45 +0100)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/libxfs-sync-6.13_2025-01-16

for you to fetch changes up to 773c4c6f33cfd0a909fb742f149bd6f59cfa62b6:

xfs: don't return an error from xfs_update_last_rtgroup_size for !XFS_RT (2025-01-16 13:27:27 -0800)

----------------------------------------------------------------
xfsprogs: new libxfs code from kernel 6.13 [1/2]

Port kernel libxfs code to userspace.

----------------------------------------------------------------
Christoph Hellwig (1):
xfs: don't return an error from xfs_update_last_rtgroup_size for !XFS_RT

libxfs/xfs_rtgroup.h | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
#13 lore raw

[GIT PULL 2/2] xfs: bug fixes for 6.13

In reply to Darrick J. Wong

Hi Andrey,

Please pull this branch with changes for xfsprogs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.

The following changes since commit 773c4c6f33cfd0a909fb742f149bd6f59cfa62b6:

xfs: don't return an error from xfs_update_last_rtgroup_size for !XFS_RT (2025-01-16 13:27:27 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/xfs-fixes-6.13_2025-01-16

for you to fetch changes up to 34738ff0ee80de73c4fd3078a89a7bef39be46b6:

mkfs: allow sizing realtime allocation groups for concurrency (2025-01-16 13:27:27 -0800)

----------------------------------------------------------------
xfs: bug fixes for 6.13 [2/2]

Bug fixes for 6.13.

----------------------------------------------------------------
Darrick J. Wong (8):
xfs_db: fix multiple dblock commands
xfs_repair: don't obliterate return codes
libxfs: fix uninit variable in libxfs_alloc_file_space
xfs_db: improve error message when unknown btree type given to btheight
mkfs: fix parsing of value-less -d/-l concurrency cli option
m4: fix statx override selection if /usr/include doesn't define it
build: initialize stack variables to zero by default
mkfs: allow sizing realtime allocation groups for concurrency

configure.ac            |   1 +
db/block.c              |   4 +-
db/btheight.c           |   6 ++
include/builddefs.in    |   2 +-
libxfs/util.c           |   2 +-
m4/package_libcdev.m4   |   2 +-
m4/package_sanitizer.m4 |  14 +++++
man/man8/mkfs.xfs.8.in  |  28 ++++++++++
mkfs/xfs_mkfs.c         | 144 ++++++++++++++++++++++++++++++++++++++++++++++--
repair/quotacheck.c     |   2 +-
10 files changed, 195 insertions(+), 10 deletions(-)