Skip to contentSkip to conversation

xfsprogs: autonomous self healing of filesystems

PATCH Superseded Darrick J. Wong 2 participants 71 messages Mar 19 – Mar 23

Subsystems (2)

v10 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

PATCH v10 1/1 xfsprogs: autonomous self healing of filesystems

Hi all,

This patchset contains the userspace and QA changes (xfs_healer) needed to put to use all the new kernel functionality to deliver live information about filesystem health events (xfs_healthmon.c) to userspace and a lot of cleanups to xfs_scrub's media verification.

In userspace, we create a new daemon program that will read the event objects and initiate repairs automatically. This daemon is managed entirely by systemd and will not block unmounting of the filesystem unless repairs are ongoing. They are auto-started by a starter service that uses fanotify.

When the patchsets under this cover letter are merged, online fsck for XFS will at long last be fully feature complete. The passive scan parts have been done since mid-2024, this final part adds proactive repair.

Here's what's left to review, thanks to Christoph for doing a bunch of xfs_healer reviews and sharing some cleanups he wanted to see in xfs_scrub; and to Zorro for merging the fstests.

[PATCHSET v10 1/2] xfsprogs: autonomous self healing of filesystems
[PATCH 19/26] xfs_healer: use statmount to find moved filesystems
[PATCHSET v10 2/2] xfs_scrub: refactor to XFS_IOC_VERIFY_MEDIA
[PATCH 01/22] libfrog: allow bitmap_free to handle a null bitmap [PATCH 02/22] mkfs: rename byte unit conversion macros [PATCH 03/22] libfrog: lift *BYTES helpers to convert.h [PATCH 04/22] xfs_scrub: report truncated devices as media errors [PATCH 05/22] xfs_scrub: fix i18n of the decode_special_owner return [PATCH 07/22] xfs_scrub: move read verification scheduling to [PATCH 09/22] xfs_scrub: don't pass the io_end_arg around everywhere [PATCH 11/22] xfs_scrub: rename nr_io_threads
[PATCH 16/22] xfs_scrub: perform media scanning of the log region [PATCH 17/22] xfs_scrub: index read-verify pools by xfs_device ids [PATCH 18/22] xfs_scrub: move failmap and other outputs into [PATCH 19/22] xfs_scrub: clean up device-related error messages [PATCH 20/22] xfs_scrub: drop SCSI_VERIFY code from disk. [PATCH 21/22] xfs_scrub: raise media verification IO limits [PATCH 22/22] xfs_scrub: allow overrides of the media verification IO

v10: cleanups of the media verification code in xfs_scrub
v9: reorg listmount/statmount, use it to find moved mounts, improve the
commit messages and documentation
v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
and filter bio status flags better
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status

--D

#1 lore raw

PATCH v10 1/2 xfsprogs: autonomous self healing of filesystems

Hi all,

This patchset builds new functionality to deliver live information about filesystem health events to userspace. This is done by creating an anonymous file that can be read() for events by userspace programs. Events are captured by hooking various parts of XFS and iomap so that metadata health failures, file I/O errors, and major changes in filesystem state (unmounts, shutdowns, etc.) can be observed by programs.

When an event occurs, the hook functions queue an event object to each event anonfd for later processing. Programs must have CAP_SYS_ADMIN to open the anonfd and there's a maximum event lag to prevent resource overconsumption. The events themselves can be read() from the anonfd as C structs for the xfs_healer daemon.

In userspace, we create a new daemon program that will read the event objects and initiate repairs automatically. This daemon is managed entirely by systemd and will not block unmounting of the filesystem unless repairs are ongoing. They are auto-started by a starter service that uses fanotify.

v10: move the xfs_scrub cleanups and changes to their own patchset,
improve the commit messages to explain why we use getmntent and statmount
v9: move listmount/statmount to libfrog; improve documentation about why
we dance with getmntent; enhance getmntent reconnection with listmount; move --svcname helpers to xfs_{healer,scrub}; improve commit messages; various tweaks to fstests
v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=health-monitoring

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=health-monitoring

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=health-monitoring
---
Commits in this patchset:
* libfrog: add a function to grab the path from an open fd and a file handle
* libfrog: create healthmon event log library functions
* libfrog: add support code for starting systemd services programmatically
* libfrog: hoist a couple of service helper functions
* libfrog: add wrappers for listmount and statmount
* man2: document the healthmon ioctl
* man2: document the media verification ioctl
* xfs_io: monitor filesystem health events
* xfs_io: add a media verify command
* xfs_healer: create daemon to listen for health events
* xfs_healer: enable repairing filesystems
* xfs_healer: use getparents to look up file names
* xfs_healer: create a per-mount background monitoring service
* xfs_healer: create a service to start the per-mount healer service
* xfs_healer: don't start service if kernel support unavailable
* xfs_healer: use the autofsck fsproperty to select mode
* xfs_healer: run full scrub after lost corruption events or targeted repair failure
* xfs_healer: use getmntent to find moved filesystems
* xfs_healer: use statmount to find moved filesystems even faster
* xfs_healer: validate that repair fds point to the monitored fs
* xfs_healer: add a manual page
* xfs_scrub: print systemd service names
* xfs_io: add listmount and statmount commands
* mkfs: enable online repair if all backrefs are enabled
* debian/control: listify the build dependencies
* debian: enable xfs_healer on the root filesystem by default

---
 healer/xfs_healer.h                            |   92 +++
 include/linux.h                                |    8 
 io/io.h                                        |    8 
 libfrog/flagmap.h                              |   23 +
 libfrog/fsproperties.h                         |    5 
 libfrog/getparents.h                           |    4 
 libfrog/healthevent.h                          |   55 ++
 libfrog/statmount.h                            |  104 ++++
 libfrog/systemd.h                              |   55 ++
 scrub/xfs_scrub.h                              |    3 
 Makefile                                       |    5 
 configure.ac                                   |   13 
 debian/control                                 |   14 -
 debian/postinst                                |    8 
 debian/prerm                                   |   13 
 debian/rules                                   |    3 
 healer/Makefile                                |   69 ++
 healer/fsrepair.c                              |  342 ++++++++++++
 healer/system-xfs_healer.slice                 |   31 +
 healer/weakhandle.c                            |  296 +++++++++++
 healer/xfs_healer.c                            |  666 ++++++++++++++++++++++++
 healer/[email protected]                  |  108 ++++
 healer/xfs_healer_start.c                      |  368 +++++++++++++
 healer/xfs_healer_start.service.in             |   85 +++
 include/builddefs.in                           |   15 +
 io/Makefile                                    |    9 
 io/healthmon.c                                 |  186 +++++++
 io/init.c                                      |    3 
 io/listmount.c                                 |  361 +++++++++++++
 io/verify_media.c                              |  180 ++++++
 libfrog/Makefile                               |   19 +
 libfrog/flagmap.c                              |   79 +++
 libfrog/getparents.c                           |   93 +++
 libfrog/healthevent.c                          |  477 +++++++++++++++++
 libfrog/statmount.c                            |   76 +++
 libfrog/systemd.c                              |  177 ++++++
 m4/package_libcdev.m4                          |  129 +++++
 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 |   75 +++
 man/man2/ioctl_xfs_health_monitor.2            |  464 +++++++++++++++++
 man/man2/ioctl_xfs_verify_media.2              |  185 +++++++
 man/man8/Makefile                              |   40 +
 man/man8/xfs_healer.8                          |  109 ++++
 man/man8/xfs_healer_start.8                    |   37 +
 man/man8/xfs_io.8                              |  133 +++++
 mkfs/xfs_mkfs.c                                |    9 
 scrub/Makefile                                 |   14 -
 scrub/xfs_scrub.c                              |   82 ++-
 47 files changed, 5272 insertions(+), 58 deletions(-)
 create mode 100644 healer/xfs_healer.h
 create mode 100644 libfrog/flagmap.h
 create mode 100644 libfrog/healthevent.h
 create mode 100644 libfrog/statmount.h
 create mode 100644 libfrog/systemd.h
 create mode 100644 debian/prerm
 create mode 100644 healer/Makefile
 create mode 100644 healer/fsrepair.c
 create mode 100644 healer/system-xfs_healer.slice
 create mode 100644 healer/weakhandle.c
 create mode 100644 healer/xfs_healer.c
 create mode 100644 healer/[email protected]
 create mode 100644 healer/xfs_healer_start.c
 create mode 100644 healer/xfs_healer_start.service.in
 create mode 100644 io/healthmon.c
 create mode 100644 io/listmount.c
 create mode 100644 io/verify_media.c
 create mode 100644 libfrog/flagmap.c
 create mode 100644 libfrog/healthevent.c
 create mode 100644 libfrog/statmount.c
 create mode 100644 libfrog/systemd.c
 create mode 100644 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2
 create mode 100644 man/man2/ioctl_xfs_health_monitor.2
 create mode 100644 man/man2/ioctl_xfs_verify_media.2
 create mode 100644 man/man8/xfs_healer.8
 create mode 100644 man/man8/xfs_healer_start.8
#2 lore raw

PATCH 1/26 libfrog: add a function to grab the path from an open fd and a file handle

From: Darrick J. Wong <[email protected]>

handle_walk_paths operates on a file handle, but requires that the fs has been registered with libhandle via path_to_fshandle. For a normal libhandle client this is the desirable behavior because the application *should* maintain an open fd to the filesystem mount.

However for xfs_healer this isn't going to work well because the healer mustn't pin the mount while it's running. It's smart enough to know how to find and reconnect to the mountpoint, but libhandle doesn't have any such concept.

Therefore, alter the libfrog getparents code so that xfs_healer can pass in the mountpoint and reconnected fd without needing libhandle. All we're really doing here is trying to obtain a user-visible path for a file that encountered problems for logging purposes; if it fails, we'll fall back to logging the inode number.

---
 libfrog/getparents.h |    4 ++
 libfrog/getparents.c |   93 ++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 82 insertions(+), 15 deletions(-)
libfrog/getparents.h +4 -0
index 8098d594219b4c..e1df30889c7606 100644
@@ -39,4 +39,8 @@ int fd_to_path(int fd, size_t ioctl_bufsize, char *path, size_t pathlen);
 int handle_to_path(const void *hanp, size_t hlen, size_t ioctl_bufsize,
 		char *path, size_t pathlen);
 
+int handle_walk_paths_fd(const char *mntpt, int mntfd, const void *hanp,
+		size_t hanlen, size_t ioctl_bufsize, walk_path_fn fn,
+		void *arg);
+
 #endif /* __LIBFROG_GETPARENTS_H_ */
libfrog/getparents.c +78 -15

141 lines not on this page · raw message

#3 lore raw

PATCH 2/26 libfrog: create healthmon event log library functions

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some helper functions to log health monitoring events so that xfs_io and xfs_healer can share logging code.

---
 libfrog/flagmap.h     |   20 +++
 libfrog/healthevent.h |   43 ++++++
 libfrog/Makefile      |    4 +
 libfrog/flagmap.c     |   62 ++++++++
 libfrog/healthevent.c |  360 +++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 489 insertions(+)
 create mode 100644 libfrog/flagmap.h
 create mode 100644 libfrog/healthevent.h
 create mode 100644 libfrog/flagmap.c
 create mode 100644 libfrog/healthevent.c
libfrog/flagmap.h new file +20 -0
new file mode 100644
index 00000000000000..8031d75a7c02a8
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025-2026 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef LIBFROG_FLAGMAP_H_
+#define LIBFROG_FLAGMAP_H_
+
+struct flag_map {
+	unsigned long long	flag;
+	const char		*string;
+};
+
+void mask_to_string(const struct flag_map *map, unsigned long long mask,
+		const char *delimiter, char *buf, size_t bufsize);
+
+const char *value_to_string(const struct flag_map *map,
+		unsigned long long value);
+
+#endif /* LIBFROG_FLAGMAP_H_ */
libfrog/healthevent.h new file +43 -0

43 lines not on this page · raw message

libfrog/Makefile +4 -0
index 927bd8d0957fab..bccd9289e5dd79 100644
@@ -19,11 +19,13 @@ bulkstat.c \
 convert.c \
 crc32.c \
 file_exchange.c \
+flagmap.c \
 fsgeom.c \
 fsproperties.c \
 fsprops.c \
 getparents.c \
 histogram.c \
+healthevent.c \
 file_attr.c \
 list_sort.c \
 linux.c \
@@ -51,11 +53,13 @@ dahashselftest.h \
 div64.h \
 fakelibattr.h \
 file_exchange.h \
+flagmap.h \
 fsgeom.h \
 fsproperties.h \
 fsprops.h \
 getparents.h \
 handle_priv.h \
+healthevent.h \
 histogram.h \
 file_attr.h \
 logging.h \
libfrog/flagmap.c new file +62 -0

62 lines not on this page · raw message

libfrog/healthevent.c new file +360 -0

360 lines not on this page · raw message

#4 lore raw

PATCH 3/26 libfrog: add support code for starting systemd services programmatically

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some simple routines for computing the name of systemd service instances and starting systemd services. These will be used by the xfs_healer_start service to start per-filesystem xfs_healer service instances.

Note that we run systemd helper programs as subprocesses for a couple of reasons. First, the path-escaping functionality is not a part of any library-accessible API, which means it can only be accessed via systemd-escape(1). Second, although the service startup functionality can be reached via dbus, doing so would introduce a new library dependency. Systemd is also undergoing a dbus -> varlink RPC transition so we avoid that mess by calling the cli systemctl(1) program.

---
 libfrog/systemd.h     |   20 ++++++
 configure.ac          |    1 
 include/builddefs.in  |    1 
 libfrog/Makefile      |    6 ++
 libfrog/systemd.c     |  177 +++++++++++++++++++++++++++++++++++++++++++++++++
 m4/package_libcdev.m4 |   19 +++++
 6 files changed, 224 insertions(+)
 create mode 100644 libfrog/systemd.h
 create mode 100644 libfrog/systemd.c
libfrog/systemd.h new file +20 -0
new file mode 100644
index 00000000000000..4f414bc3c1e9c3
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2026 Oracle.  All rights reserved.
+ * All Rights Reserved.
+ */
+#ifndef __LIBFROG_SYSTEMD_H__
+#define __LIBFROG_SYSTEMD_H__
+
+int systemd_path_instance_unit_name(const char *unit_template,
+		const char *path, char *unitname, size_t unitnamelen);
+
+enum systemd_unit_manage {
+	UM_STOP,
+	UM_START,
+	UM_RESTART,
+};
+
+int systemd_manage_unit(enum systemd_unit_manage how, const char *unitname);
+
+#endif /* __LIBFROG_SYSTEMD_H__ */
configure.ac +1 -0
index 8092b8656ef94b..a9febabc71cfc7 100644
@@ -182,6 +182,7 @@ AC_CONFIG_UDEV_RULE_DIR
 AC_HAVE_BLKID_TOPO
 AC_HAVE_TRIVIAL_AUTO_VAR_INIT
 AC_STRERROR_R_RETURNS_STRING
+AC_HAVE_CLOSE_RANGE
 
 if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then
         AC_PACKAGE_CHECK_UBSAN
include/builddefs.in +1 -0
index b38a099b7d525a..4a2cb757c0bdb3 100644
@@ -118,6 +118,7 @@ HAVE_UDEV = @have_udev@
 UDEV_RULE_DIR = @udev_rule_dir@
 HAVE_LIBURCU_ATOMIC64 = @have_liburcu_atomic64@
 STRERROR_R_RETURNS_STRING = @strerror_r_returns_string@
+HAVE_CLOSE_RANGE = @have_close_range@
 
 GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall
 #	   -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
libfrog/Makefile +6 -0
index bccd9289e5dd79..89a0332ae85372 100644
@@ -36,6 +36,7 @@ ptvar.c \
 radix-tree.c \
 randbytes.c \
 scrub.c \
+systemd.c \
 util.c \
 workqueue.c \
 zones.c
@@ -70,6 +71,7 @@ radix-tree.h \
 randbytes.h \
 scrub.h \
 statx.h \
+systemd.h \
 workqueue.h \
 zones.h
 
@@ -90,6 +92,10 @@ ifeq ($(HAVE_GETRANDOM_NONBLOCK),yes)
 LCFLAGS += -DHAVE_GETRANDOM_NONBLOCK
 endif
 
+ifeq ($(HAVE_CLOSE_RANGE),yes)
+CFLAGS += -DHAVE_CLOSE_RANGE
+endif
+
 default: ltdepend $(LTLIBRARY) $(GETTEXT_PY)
 
 crc32table.h: gen_crc32table.c crc32defs.h
libfrog/systemd.c new file +177 -0

177 lines not on this page · raw message

m4/package_libcdev.m4 +19 -0
index c5538c30d2518a..b3d87229d3367a 100644
@@ -347,3 +347,22 @@ puts(strerror_r(0, buf, sizeof(buf)));
     CFLAGS="$OLD_CFLAGS"
     AC_SUBST(strerror_r_returns_string)
   ])
+
+#
+# Check if close_range exists
+#
+AC_DEFUN([AC_HAVE_CLOSE_RANGE],
+  [AC_MSG_CHECKING([for close_range])
+    AC_LINK_IFELSE(
+    [AC_LANG_PROGRAM([[
+#define _GNU_SOURCE
+#include <unistd.h>
+#include <linux/close_range.h>
+  ]], [[
+close_range(0, 0, 0);
+  ]])
+    ], have_close_range=yes
+       AC_MSG_RESULT(yes),
+       AC_MSG_RESULT(no))
+    AC_SUBST(have_close_range)
+  ])
#5 lore raw

PATCH 4/26 libfrog: hoist a couple of service helper functions

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Hoist a couple of service/daemon-related helper functions to libfrog so that we can share the code between xfs_scrub and xfs_healer.

---
 libfrog/systemd.h |   28 ++++++++++++++++++++++++++++
 scrub/xfs_scrub.c |   32 +++++++++-----------------------
 2 files changed, 37 insertions(+), 23 deletions(-)
libfrog/systemd.h +28 -0
index 4f414bc3c1e9c3..c96df4afa39aa6 100644
@@ -17,4 +17,32 @@ enum systemd_unit_manage {
 
 int systemd_manage_unit(enum systemd_unit_manage how, const char *unitname);
 
+static inline bool systemd_is_service(void)
+{
+	return getenv("SERVICE_MODE") != NULL;
+}
+
+/* Special processing for a service/daemon program that is exiting. */
+static inline int
+systemd_service_exit(int ret)
+{
+	/*
+	 * We have to sleep 2 seconds here because journald uses the pid to
+	 * connect our log messages to the systemd service.  This is critical
+	 * for capturing all the log messages if the service fails, because
+	 * failure analysis tools use the service name to gather log messages
+	 * for reporting.
+	 */
+	sleep(2);
+
+	/*
+	 * If we're being run as a service, the return code must fit the LSB
+	 * init script action error guidelines, which is to say that we
+	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
+	 * section 22.2) and hope the admin will scan the log for what actually
+	 * happened.
+	 */
+	return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+}
+
 #endif /* __LIBFROG_SYSTEMD_H__ */
scrub/xfs_scrub.c +9 -23
index 3dba972a7e8d2a..79937aa8cce4c4 100644
@@ -19,6 +19,7 @@
 #include "unicrash.h"
 #include "progress.h"
 #include "libfrog/histogram.h"
+#include "libfrog/systemd.h"
 
 /*
  * XFS Online Metadata Scrub (and Repair)
@@ -866,8 +867,7 @@ main(
 	if (stdout_isatty && !progress_fp)
 		progress_fp = fdopen(1, "w+");
 
-	if (getenv("SERVICE_MODE"))
-		is_service = true;
+	is_service = systemd_is_service();
 
 	/* Initialize overall phase stats. */
 	error = phase_start(&all_pi, 0, NULL);
@@ -960,29 +960,15 @@ main(
 	hist_free(&ctx.datadev_hist);
 	hist_free(&ctx.rtdev_hist);
 
-	/*
-	 * If we're being run as a service, the return code must fit the LSB
-	 * init script action error guidelines, which is to say that we
-	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
-	 * section 22.2) and hope the admin will scan the log for what
-	 * actually happened.
-	 *
-	 * We have to sleep 2 seconds here because journald uses the pid to
-	 * connect our log messages to the systemd service.  This is critical
-	 * for capturing all the log messages if the scrub fails, because the
-	 * fail service uses the service name to gather log messages for the
-	 * error report.
-	 *
-	 * Note: We don't count a lack of kernel support as a service failure
-	 * because we haven't determined that there's anything wrong with the
-	 * filesystem.
-	 */
 	if (is_service) {
-		sleep(2);
+		/*
+		 * Note: We don't count a lack of kernel support as a service
+		 * failure because we haven't determined that there's anything
+		 * wrong with the filesystem.
+		 */
 		if (!ctx.scrub_setup_succeeded)
-			return 0;
-		if (ret != SCRUB_RET_SUCCESS)
-			return 1;
+			ret = 0;
+		return systemd_service_exit(ret);
 	}
 
 	return ret;
#6 lore raw

PATCH 5/26 libfrog: add wrappers for listmount and statmount

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some wrappers for listmount and statmount so that we don't have to open-code the kernel ABI quirks in every utility program that uses it. Note that glibc seems to have discussed providing a wrapper in late 2023 but took no action; and the listmount manpage says that there is no glibc wrapper.

---
 include/linux.h       |    8 +++-
 libfrog/statmount.h   |  104 +++++++++++++++++++++++++++++++++++++++++++++++++
 configure.ac          |    5 ++
 include/builddefs.in  |    7 +++
 libfrog/Makefile      |    9 ++++
 libfrog/statmount.c   |   76 ++++++++++++++++++++++++++++++++++++
 m4/package_libcdev.m4 |   86 +++++++++++++++++++++++++++++++++++++++++
 7 files changed, 294 insertions(+), 1 deletion(-)
 create mode 100644 libfrog/statmount.h
 create mode 100644 libfrog/statmount.c
include/linux.h +7 -1
index 3ea9016272e688..8972c9596c75f5 100644
@@ -32,7 +32,13 @@
 #ifdef OVERRIDE_SYSTEM_FSXATTR
 # define fsxattr sys_fsxattr
 #endif
-#include <linux/fs.h> /* fsxattr defintion for new kernels */
+#ifdef OVERRIDE_SYSTEM_STATMOUNT
+# define statmount sys_statmount
+#endif
+#include <linux/fs.h> /* fsxattr/statmount defintion for new kernels */
+#ifdef OVERRIDE_SYSTEM_STATMOUNT
+# undef statmount
+#endif
 #ifdef OVERRIDE_SYSTEM_FSXATTR
 # undef fsxattr
 #endif
libfrog/statmount.h new file +104 -0

104 lines not on this page · raw message

configure.ac +5 -0
index a9febabc71cfc7..cffcaf373cfa5e 100644
@@ -183,6 +183,11 @@ AC_HAVE_BLKID_TOPO
 AC_HAVE_TRIVIAL_AUTO_VAR_INIT
 AC_STRERROR_R_RETURNS_STRING
 AC_HAVE_CLOSE_RANGE
+AC_HAVE_LISTMOUNT
+if test "$have_listmount" = "yes"; then
+	AC_HAVE_LISTMOUNT_NS_FD
+	AC_HAVE_STATMOUNT_SUPPORTED_MASK
+fi
 
 if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then
         AC_PACKAGE_CHECK_UBSAN
include/builddefs.in +7 -0
index 4a2cb757c0bdb3..d2d25c8a0ed676 100644
@@ -119,6 +119,10 @@ UDEV_RULE_DIR = @udev_rule_dir@
 HAVE_LIBURCU_ATOMIC64 = @have_liburcu_atomic64@
 STRERROR_R_RETURNS_STRING = @strerror_r_returns_string@
 HAVE_CLOSE_RANGE = @have_close_range@
+HAVE_LISTMOUNT = @have_listmount@
+HAVE_LISTMOUNT_NS_FD = @have_listmount_ns_fd@
+HAVE_STATMOUNT_SUPPORTED_MASK = @have_statmount_supported_mask@
+NEED_INTERNAL_STATMOUNT = @need_internal_statmount@
 
 GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall
 #	   -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
@@ -141,6 +145,9 @@ endif
 ifeq ($(NEED_INTERNAL_STATX),yes)
 PCFLAGS+= -DOVERRIDE_SYSTEM_STATX
 endif
+ifeq ($(NEED_INTERNAL_STATMOUNT),yes)
+PCFLAGS+= -DOVERRIDE_SYSTEM_STATMOUNT
+endif
 ifeq ($(HAVE_GETFSMAP),yes)
 PCFLAGS+= -DHAVE_GETFSMAP
 endif
libfrog/Makefile +9 -0
index 89a0332ae85372..22668212f22b93 100644
@@ -96,6 +96,15 @@ ifeq ($(HAVE_CLOSE_RANGE),yes)
 CFLAGS += -DHAVE_CLOSE_RANGE
 endif
 
+ifeq ($(HAVE_LISTMOUNT),yes)
+CFILES += statmount.c
+HFILES += statmount.h
+endif
+
+ifeq ($(HAVE_LISTMOUNT_NS_FD),yes)
+CFLAGS+=-DHAVE_LISTMOUNT_NS_FD
+endif
+
 default: ltdepend $(LTLIBRARY) $(GETTEXT_PY)
 
 crc32table.h: gen_crc32table.c crc32defs.h
libfrog/statmount.c new file +76 -0

76 lines not on this page · raw message

m4/package_libcdev.m4 +86 -0

89 lines not on this page · raw message

#7 lore raw

PATCH 6/26 man2: document the healthmon ioctl

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Document the XFS_IOC_HEALTH_MONITOR and
XFS_IOC_HEALTH_FD_ON_MONITORED_FS ioctls.

---
 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 |   75 ++++
 man/man2/ioctl_xfs_health_monitor.2            |  464 ++++++++++++++++++++++++
 2 files changed, 539 insertions(+)
 create mode 100644 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2
 create mode 100644 man/man2/ioctl_xfs_health_monitor.2
man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 new file +75 -0
new file mode 100644
index 00000000000000..bbc5ce9bbabf53
@@ -0,0 +1,75 @@
+.\" Copyright (c) 2025-2026, Oracle.  All rights reserved.
+.\"
+.\" %%%LICENSE_START(GPLv2+_DOC_FULL)
+.\" SPDX-License-Identifier: GPL-2.0+
+.\" %%%LICENSE_END
+.TH IOCTL-XFS-HEALTH-FD-ON-MONITORED-FS 2 2026-01-04 "XFS"
+.SH NAME
+ioctl_xfs_health_fd_on_monitored_fs \- check if the given fd belongs to the same fs being monitored
+.SH SYNOPSIS
+.br
+.B #include <xfs/xfs_fs.h>
+.PP
+.BI "int ioctl(int " healthmon_fd ", XFS_IOC_HEALTH_FD_ON_MONITORED_FS, struct xfs_health_file_on_monitored_fs *" arg );
+.SH DESCRIPTION
+This XFS healthmon fd ioctl asks the kernel driver if the file descriptor
+passed in via
+.I arg
+points to a file on the same filesystem that is being monitored by
+.IR healthmon_fd .
+The file descriptor is conveyed in a structure of the following form:
+.PP
+.in +4n
+.nf
+struct xfs_health_file_on_monitored_fs {
+	__s32 fd;
+	__u32 flags;
+};
+.fi
+.in
+.PP
+The field
+.I flags
+must be zero.
+.PP
+The field
+.I fd
+is a descriptor of an open file.
+.PP
+The argument
+.I healthmon_fd
+must be a file opened via the
+.B XFS_IOC_HEALTH_MONITOR
+ioctl.
+.SH RETURN VALUE
+On error, \-1 is returned, and
+.I errno
+is set to indicate the error.
+If the file descriptor points to a file on the same filesystem that is being
+monitored, 0 is returned.
+.PP
+.SH ERRORS
+Error codes can be one of, but are not limited to, the following:
+.TP
+.B ESTALE
+The open file is not on the same filesystem that is being monitored.
+.TP
+.B EINVAL
+One or more of the arguments specified is invalid.
+.TP
+.B EBADF
+.I arg.fd
+does not refer to an open file.
+.TP
+.B EFAULT
+The
+.I arg
+structure could not be copied into the kernel.
+.TP
+.B ENOTTY
+.I healthmon_fd
+is not a XFS health monitoring file.
+.SH CONFORMING TO
+This API is specific to XFS filesystem on the Linux kernel.
+.SH SEE ALSO
+.BR ioctl_xfs_health_monitor (2)
man/man2/ioctl_xfs_health_monitor.2 new file +464 -0

464 lines not on this page · raw message

#8 lore raw

PATCH 7/26 man2: document the media verification ioctl

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Document XFS_IOC_VERIFY_MEDIA, which is a new ioctl for xfs_scrub to perform media scans on the disks underneath the filesystem. This will enable media errors to be reported to xfs_healer and fsnotify.

---
 man/man2/ioctl_xfs_verify_media.2 |  185 +++++++++++++++++++++++++++++++++++++
 1 file changed, 185 insertions(+)
 create mode 100644 man/man2/ioctl_xfs_verify_media.2
man/man2/ioctl_xfs_verify_media.2 new file +185 -0
new file mode 100644
index 00000000000000..bd0d4579f5a364
@@ -0,0 +1,185 @@
+.\" Copyright (c) 2025-2026, Oracle.  All rights reserved.
+.\"
+.\" %%%LICENSE_START(GPLv2+_DOC_FULL)
+.\" SPDX-License-Identifier: GPL-2.0+
+.\" %%%LICENSE_END
+.TH IOCTL-XFS-VERIFY-MEDIA 2 2026-01-09 "XFS"
+.SH NAME
+ioctl_xfs_verify_media \- verify the media of the devices backing XFS
+.SH SYNOPSIS
+.br
+.B #include <xfs/xfs_fs.h>
+.PP
+.BI "int ioctl(int " fd ", XFS_IOC_VERIFY_MEDIA, struct xfs_verify_media *" arg );
+.SH DESCRIPTION
+Verify the media of a storage device backing an XFS filesystem.
+If errors are found, report the error to the kernel so that it can generate
+health events for the health monitoring system and fsnotify.
+The verification request is conveyed in a structure of the following form:
+.PP
+.in +4n
+.nf
+struct xfs_verify_error {
+	__u32	me_dev;
+	__u32	me_flags;
+	__u64	me_start_daddr;
+	__u64	me_end_daddr;
+	__u32	me_ioerror;
+	__u32	me_pad;
+};
+.fi
+.in
+.PP
+The field
+.I me_pad
+must be zero.
+.PP
+The field
+.I me_ioerror
+will be set if the ioctl returns success.
+.PP
+The fields
+.I me_start_daddr
+and
+.I me_end_daddr
+are the range of the storage device to verify.
+Both values must be in units of 512-byte blocks.
+The
+.I me_start_daddr
+field is inclusive, and the
+.I me_end_daddr
+field is exclusive.
+If
+.I me_end_daddr
+is larger than the size of the device, the kernel will set it to the size of
+the device.
+
+If the system call returns success and any part of the storage device range was
+successfully verified, the
+.I me_start_daddr
+field will be updated to reflect the successful verification.
+If after this update the
+.I me_start_daddr
+is equal to
+.IR me_end_daddr ,
+then the entire range was verified successfully.
+
+If not, then a media error was encountered and the caller should generate a
+series of secondary calls to this ioctl with smaller ranges to discover the
+exact location and type of media error.
+The type of media error will be written to the
+.I me_ioerror
+field.
+
+.PP
+The field
+.I me_dev
+must be one of the following values:
+.RS 0.4i
+.TP
+.B XFS_DEV_DATA
+Verify the data device.
+.TP
+.B XFS_DEV_LOG
+Verify the external log device.
+.TP
+.B XFS_DEV_RT
+Verify the realtime device.
+.RE
+.PP
+The field
+.I me_flags
+is a bitmask of one of the following values:
+.RS 0.4i
+.TP
+.B XFS_VERIFY_MEDIA_REPORT
+Report all media errors to fsnotify.
+.RE
+
+The
+.IR me_max_io_size
+field, if nonzero, will be used as advice for the maximum size of the IO to
+send to the device.
+
+The
+.I me_rest_us
+field will cause the kernel to pause for this many microseconds between IO
+requests.
+
+.SH RETURN VALUE
+On runtime error, \-1 is returned, and
+.I errno
+is set to indicate the error.
+If 0 is returned, then
+.I start_daddr
+or
+.I ioerror
+will be updated.
+.PP
+.SH ERRORS
+Error codes can be one of, but are not limited to, the following:
+.TP
+.B EPERM
+The calling process does not have sufficient privilege.
+.TP
+.B EINVAL
+One or more of the arguments specified is invalid.
+.TP
+.B EFAULT
+The
+.I arg
+structure could not be copied into the kernel.
+.TP
+.B ENODEV
+The device is not present.
+.TP
+.B ENOMEM
+There was not enough memory to perform the verification.
+
+.SH I/O ERRORS
+The
+.I ioerror
+field could be set to one of the following:
+.TP
+.B 0
+The verification I/O succeeded.
+.TP
+.B EOPNOTSUPP
+.TP
+.B ETIMEDOUT
+The kernel timed out the verification I/O command.
+.TP
+.B ENOLINK
+The transportation link to the storage device was down temporarily.
+.TP
+.B EREMOTEIO
+The storage target controller suffered a critical error.
+.TP
+.B ENODATA
+The storage target media suffered a critical error.
+.TP
+.B EILSEQ
+Storage protection metadata did not validate successfully.
+.TP
+.B ENOMEM
+There was not enough memory to allocate an I/O request.
+.TP
+.B ENODEV
+The storage device is offline.
+.TP
+.B ETIME
+The storage device timed out the I/O command.
+.TP
+.B EINVAL
+The I/O request was rejected by the device for being invalid.
+.TP
+.B EIO
+An I/O error occurred but no specific details are available.
+.RE
+.PP
+This list is not exhaustive and may grow in the future.
+
+.SH CONFORMING TO
+This API is specific to XFS filesystem on the Linux kernel.
+.SH SEE ALSO
+.BR ioctl_xfs_health_monitor (2)
#9 lore raw

PATCH 8/26 xfs_io: monitor filesystem health events

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a subcommand to monitor for health events generated by the kernel.

---
 io/io.h           |    1 
 io/Makefile       |    1 
 io/healthmon.c    |  186 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 io/init.c         |    1 
 man/man8/xfs_io.8 |   25 +++++++
 5 files changed, 214 insertions(+)
 create mode 100644 io/healthmon.c
io/io.h +1 -0
index 35fb8339eeb5aa..2f5262bce6acbb 100644
@@ -162,3 +162,4 @@ extern void		bulkstat_init(void);
 void			exchangerange_init(void);
 void			fsprops_init(void);
 void			aginfo_init(void);
+void			healthmon_init(void);
io/Makefile +1 -0
index 444e2d6a557d5d..8e3783353a52b5 100644
@@ -25,6 +25,7 @@ CFILES = \
 	fsuuid.c \
 	fsync.c \
 	getrusage.c \
+	healthmon.c \
 	imap.c \
 	init.c \
 	inject.c \
io/healthmon.c new file +186 -0

186 lines not on this page · raw message

io/init.c +1 -0
index 49e9e7cb88214b..cb5573f45ccfbc 100644
@@ -92,6 +92,7 @@ init_commands(void)
 	crc32cselftest_init();
 	exchangerange_init();
 	fsprops_init();
+	healthmon_init();
 }
 
 /*
man/man8/xfs_io.8 +25 -0
index 0a673322fde3a1..f7f2956a54a7aa 100644
@@ -1356,6 +1356,31 @@ .SH FILESYSTEM COMMANDS
 .B thaw
 Undo the effects of a filesystem freeze operation.
 Only available in expert mode and requires privileges.
+.TP
+.BI "healthmon [ \-c " bufsize " ] [ \-c ] [ \-d " delay_ms " ] [ \-p ] [ \-v ]"
+Watch for filesystem health events and write them to the console.
+.RE
+.RS 1.0i
+.PD 0
+.TP
+.BI "\-b " bufsize
+Use a buffer of this size to read events from the kernel.
+.TP
+.BI \-c
+Close the open file and replace it with the monitor file.
+.TP
+.BI "\-d " delay_ms
+Sleep for this long between read attempts.
+.TP
+.B \-p
+Probe for the existence of the functionality by opening the monitoring fd and
+closing it immediately.
+.TP
+.BI \-v
+Request all health events, even if nothing changed.
+.PD
+.RE
+
 .TP
 .BI "inject [ " tag " ]"
 Inject errors into a filesystem to observe filesystem behavior at
#10 lore raw

PATCH 9/26 xfs_io: add a media verify command

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a subcommand to invoke the media verification ioctl to make sure that we can actually check the storage underneath an xfs filesystem.

---
 io/io.h           |    1 
 io/Makefile       |    3 +
 io/init.c         |    1 
 io/verify_media.c |  180 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_io.8 |   42 ++++++++++++
 5 files changed, 226 insertions(+), 1 deletion(-)
 create mode 100644 io/verify_media.c
io/io.h +1 -0
index 2f5262bce6acbb..0f12b3cfed5e76 100644
@@ -163,3 +163,4 @@ void			exchangerange_init(void);
 void			fsprops_init(void);
 void			aginfo_init(void);
 void			healthmon_init(void);
+void			verifymedia_init(void);
io/Makefile +2 -1
index 8e3783353a52b5..79d5e172b8f31f 100644
@@ -51,7 +51,8 @@ CFILES = \
 	sync.c \
 	sync_file_range.c \
 	truncate.c \
-	utimes.c
+	utimes.c \
+	verify_media.c
 
 LLDLIBS = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG) $(LIBPTHREAD) $(LIBUUID)
 LTDEPENDENCIES = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG)
io/init.c +1 -0
index cb5573f45ccfbc..f2a551ef559200 100644
@@ -93,6 +93,7 @@ init_commands(void)
 	exchangerange_init();
 	fsprops_init();
 	healthmon_init();
+	verifymedia_init();
 }
 
 /*
io/verify_media.c new file +180 -0

180 lines not on this page · raw message

man/man8/xfs_io.8 +42 -0

48 lines not on this page · raw message

#11 lore raw

PATCH 10/26 xfs_healer: create daemon to listen for health events

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a daemon program that can listen for and log health events. Eventually this will be used to self-heal filesystems in real time.

Because events can take a while to process, the main thread reads event objects from the healthmon fd and dispatches them to a background workqueue as quickly as it can. This split of responsibilities is necessary because the kernel event queue will drop events if the queue fills up, and each event can take some time to process (logging, repairs, etc.) so we don't want to lose events.

To be clear, xfs_healer and xfs_scrub are complementary tools:

Scrub walks the whole filesystem, finds stuff that needs fixing or rebuilding, and rebuilds it. This is sort of analogous to a patrol scrub.

Healer listens for metadata corruption messages from the kernel and issues a targeted repair of that structure. This is kind of like an ondemand scrub.

My end goal is that xfs_healer (the service) is active all the time and can respond instantly to a corruption report, whereas xfs_scrub (the service) gets run periodically as a cron job.

xfs_healer can decide that it's overwhelmed with problems and start xfs_scrub to deal with the mess. Ideally you don't crash the filesystem and then have to use xfs_repair to smash your way back to a mountable filesystem.

By default we run xfs_healer as a background service, which means that we only start two threads -- one to read the events, and another to process them. In other words, we try not to use all available hardware resources for repairs. The foreground mode switch starts up a large number of threads to try to increase parallelism, which may or may not be useful for repairs depending on how much metadata the kernel needs to scan.

---
 healer/xfs_healer.h  |   47 ++++++
 Makefile             |    5 +
 configure.ac         |    6 +
 healer/Makefile      |   35 ++++
 healer/xfs_healer.c  |  391 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/builddefs.in |    1 
 6 files changed, 485 insertions(+)
 create mode 100644 healer/xfs_healer.h
 create mode 100644 healer/Makefile
 create mode 100644 healer/xfs_healer.c
healer/xfs_healer.h new file +47 -0
new file mode 100644
index 00000000000000..bcddde5db0cc47
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025-2026 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef XFS_HEALER_XFS_HEALER_H_
+#define XFS_HEALER_XFS_HEALER_H_
+
+extern char *progname;
+
+/*
+ * When running in environments with restrictive security policies, healer
+ * might not be allowed to access the global mount tree.  However, processes
+ * are usually still allowed to see their own mount tree, so use this path for
+ * all mount table queries.
+ */
+#define _PATH_PROC_MOUNTS	"/proc/self/mounts"
+
+struct healer_ctx {
+	/* CLI options, must be int */
+	int			debug;
+	int			log;
+	int			everything;
+	int			foreground;
+
+	/* fd and fs geometry for mount */
+	struct xfs_fd		mnt;
+
+	/* Shared reference to the user's mountpoint for logging */
+	const char		*mntpoint;
+
+	/* Shared reference to the getmntent fsname for reconnecting */
+	const char		*fsname;
+
+	/* file stream of monitor and buffer */
+	FILE			*mon_fp;
+	char			*mon_buf;
+
+	/* coordinates logging printfs */
+	pthread_mutex_t		conlock;
+
+	/* event queue */
+	struct workqueue	event_queue;
+	bool			queue_active;
+};
+
+#endif /* XFS_HEALER_XFS_HEALER_H_ */
Makefile +5 -0
index c73aa391bc5f43..1f499c30f3457e 100644
@@ -69,6 +69,10 @@ ifeq ("$(ENABLE_SCRUB)","yes")
 TOOL_SUBDIRS += scrub
 endif
 
+ifeq ("$(ENABLE_HEALER)","yes")
+TOOL_SUBDIRS += healer
+endif
+
 ifneq ("$(XGETTEXT)","")
 TOOL_SUBDIRS += po
 endif
@@ -100,6 +104,7 @@ mkfs: libxcmd
 spaceman: libxcmd libhandle
 scrub: libhandle libxcmd
 rtcp: libfrog
+healer: libhandle
 
 ifeq ($(HAVE_BUILDDEFS), yes)
 include $(BUILDRULES)
configure.ac +6 -0
index cffcaf373cfa5e..90af1f84035ee6 100644
@@ -110,6 +110,12 @@ AC_ARG_ENABLE(libicu,
 [  --enable-libicu=[yes/no]  Enable Unicode name scanning in xfs_scrub (libicu) [default=probe]],,
 	enable_libicu=probe)
 
+# Enable xfs_healer build
+AC_ARG_ENABLE(healer,
+[  --enable-healer=[yes/no]  Enable build of xfs_healer utility [[default=yes]]],,
+	enable_healer=yes)
+AC_SUBST(enable_healer)
+
 #
 # If the user specified a libdir ending in lib64 do not append another
 # 64 to the library names.
healer/Makefile new file +35 -0
new file mode 100644
index 00000000000000..e82c820883669a
@@ -0,0 +1,35 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024-2026 Oracle.  All Rights Reserved.
+#
+
+TOPDIR = ..
+builddefs=$(TOPDIR)/include/builddefs
+include $(builddefs)
+
+INSTALL_HEALER = install-healer
+
+LTCOMMAND = xfs_healer
+
+CFILES = \
+xfs_healer.c
+
+HFILES = \
+xfs_healer.h
+
+LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD)
+LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG)
+LLDFLAGS = -static
+
+default: depend $(LTCOMMAND)
+
+include $(BUILDRULES)
+
+install: $(INSTALL_HEALER)
+
+install-healer: default
+	$(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR)
+	$(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR)
+
+install-dev:
+
+-include .dep
healer/xfs_healer.c new file +391 -0

391 lines not on this page · raw message

include/builddefs.in +1 -0
index d2d25c8a0ed676..0ab2bf1702f0f0 100644
@@ -91,6 +91,7 @@ ENABLE_SHARED	= @enable_shared@
 ENABLE_GETTEXT	= @enable_gettext@
 ENABLE_EDITLINE	= @enable_editline@
 ENABLE_SCRUB	= @enable_scrub@
+ENABLE_HEALER	= @enable_healer@
 
 HAVE_ZIPPED_MANPAGES = @have_zipped_manpages@
 
#12 lore raw

PATCH 11/26 xfs_healer: enable repairing filesystems

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make it so that our health monitoring daemon can initiate repairs in response to reports of corrupt filesystem metadata. Repairs are initiated from the background workers as explained in the previous patch.

Note that just like xfs_scrub, xfs_healer's ability to repair metadata relies heavily on back references such as reverse mappings and directory parent pointers to add redundancy to the filesystem. Check for these two features and whine a bit if they are missing, just like scrub.

There's a bit of trickery with the fd that is used to initiate repairs in the kernel. Because an open fd will pin the filesystem in memory, xfs_healer can only hold an open fd to the target filesystem while it's performing repairs. Therefore, at startup xfs_healer must sample enough information about the target filesystem to reconnect to it later on. Currently, the fs source (aka the data device path) and the root directory handle are sufficient to do this.

Someday we might be able to have revocable fds, which would eliminate the need for such efforts in userspace.

---
 healer/xfs_healer.h   |   28 ++++++
 libfrog/flagmap.h     |    3 +
 libfrog/healthevent.h |   12 ++
 healer/Makefile       |    2 
 healer/fsrepair.c     |  249 +++++++++++++++++++++++++++++++++++++++++++++++++
 healer/weakhandle.c   |  115 +++++++++++++++++++++++
 healer/xfs_healer.c   |   55 +++++++++++
 libfrog/flagmap.c     |   17 +++
 libfrog/healthevent.c |  117 +++++++++++++++++++++++
 9 files changed, 598 insertions(+)
 create mode 100644 healer/fsrepair.c
 create mode 100644 healer/weakhandle.c
healer/xfs_healer.h +28 -0
index bcddde5db0cc47..a4de1ad32a408f 100644
@@ -8,6 +8,9 @@
 
 extern char *progname;
 
+struct weakhandle;
+struct hme_prefix;
+
 /*
  * When running in environments with restrictive security policies, healer
  * might not be allowed to access the global mount tree.  However, processes
@@ -22,6 +25,7 @@ struct healer_ctx {
 	int			log;
 	int			everything;
 	int			foreground;
+	int			want_repair;
 
 	/* fd and fs geometry for mount */
 	struct xfs_fd		mnt;
@@ -32,6 +36,9 @@ struct healer_ctx {
 	/* Shared reference to the getmntent fsname for reconnecting */
 	const char		*fsname;
 
+	/* weak file handle so we can reattach to filesystem */
+	struct weakhandle	*wh;
+
 	/* file stream of monitor and buffer */
 	FILE			*mon_fp;
 	char			*mon_buf;
@@ -44,4 +51,25 @@ struct healer_ctx {
 	bool			queue_active;
 };
 
+static inline bool healer_has_rmapbt(const struct healer_ctx *ctx)
+{
+	return ctx->mnt.fsgeom.flags & XFS_FSOP_GEOM_FLAGS_RMAPBT;
+}
+
+static inline bool healer_has_parent(const struct healer_ctx *ctx)
+{
+	return ctx->mnt.fsgeom.flags & XFS_FSOP_GEOM_FLAGS_PARENT;
+}
+
+/* repair.c */
+int repair_metadata(struct healer_ctx *ctx, const struct hme_prefix *pfx,
+		const struct xfs_health_monitor_event *hme);
+bool healer_can_repair(struct healer_ctx *ctx);
+
+/* weakhandle.c */
+int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
+		struct weakhandle **whp);
+int weakhandle_reopen(struct weakhandle *wh, int *fd);
+void weakhandle_free(struct weakhandle **whp);
+
 #endif /* XFS_HEALER_XFS_HEALER_H_ */
libfrog/flagmap.h +3 -0
index 8031d75a7c02a8..05110c3544dc97 100644
@@ -14,6 +14,9 @@ struct flag_map {
 void mask_to_string(const struct flag_map *map, unsigned long long mask,
 		const char *delimiter, char *buf, size_t bufsize);
 
+const char *lowest_set_mask_string(const struct flag_map *map,
+		unsigned long long mask);
+
 const char *value_to_string(const struct flag_map *map,
 		unsigned long long value);
 
libfrog/healthevent.h +12 -0
index 6de41bc797100c..4f3c8ba639ec4c 100644
@@ -40,4 +40,16 @@ hme_prefix_init(
 void hme_report_event(const struct hme_prefix *pfx,
 		const struct xfs_health_monitor_event *hme);
 
+enum repair_outcome {
+	REPAIR_SUCCESS,
+	REPAIR_FAILED,
+	REPAIR_PROBABLY_OK,
+	REPAIR_UNNECESSARY,
+};
+
+void report_health_repair(const struct hme_prefix *pfx,
+		const struct xfs_health_monitor_event *hme,
+		uint32_t event_mask,
+		enum repair_outcome outcome);
+
 #endif /* LIBFROG_HEALTHEVENT_H_ */
healer/Makefile +2 -0
index e82c820883669a..981192b81af626 100644
@@ -11,6 +11,8 @@ INSTALL_HEALER = install-healer
 LTCOMMAND = xfs_healer
 
 CFILES = \
+fsrepair.c \
+weakhandle.c \
 xfs_healer.c
 
 HFILES = \
healer/fsrepair.c new file +249 -0

249 lines not on this page · raw message

healer/weakhandle.c new file +115 -0

115 lines not on this page · raw message

healer/xfs_healer.c +55 -0

103 lines not on this page · raw message

libfrog/flagmap.c +17 -0
index 631c4bbc8f1dc0..ce413297780a2a 100644
@@ -44,6 +44,23 @@ mask_to_string(
 		snprintf(buf, bufsize, "%s0x%llx", tag, mask & ~seen);
 }
 
+/*
+ * Given a mapping of bits to strings and a bitmask, return the string
+ * corresponding to the lowest set bit in the mask.
+ */
+const char *
+lowest_set_mask_string(
+	const struct flag_map	*map,
+	unsigned long long	mask)
+{
+	for (; map->string; map++) {
+		if (mask & map->flag)
+			return _(map->string);
+	}
+
+	return _("unknown flag");
+}
+
 /*
  * Given a mapping of values to strings and a value, return the matching string
  * or confusion.
libfrog/healthevent.c +117 -0

120 lines not on this page · raw message

#13 lore raw

PATCH 12/26 xfs_healer: use getparents to look up file names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the kernel tells about something that happened to a file, use the GETPARENTS ioctl to try to look up the path to that file for more ergonomic reporting.

---
 healer/xfs_healer.h |    6 ++++
 healer/fsrepair.c   |   16 ++++++++-
 healer/weakhandle.c |   86 +++++++++++++++++++++++++++++++++++++++++++++++++++
 healer/xfs_healer.c |   45 ++++++++++++++++++++++++++-
 4 files changed, 149 insertions(+), 4 deletions(-)
healer/xfs_healer.h +6 -0
index a4de1ad32a408f..6d12921245934c 100644
@@ -61,6 +61,10 @@ static inline bool healer_has_parent(const struct healer_ctx *ctx)
 	return ctx->mnt.fsgeom.flags & XFS_FSOP_GEOM_FLAGS_PARENT;
 }
 
+void lookup_path(struct healer_ctx *ctx,
+		const struct xfs_health_monitor_event *hme,
+		struct hme_prefix *pfx);
+
 /* repair.c */
 int repair_metadata(struct healer_ctx *ctx, const struct hme_prefix *pfx,
 		const struct xfs_health_monitor_event *hme);
@@ -71,5 +75,7 @@ int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
 		struct weakhandle **whp);
 int weakhandle_reopen(struct weakhandle *wh, int *fd);
 void weakhandle_free(struct weakhandle **whp);
+int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
+		char *path, size_t pathlen);
 
 #endif /* XFS_HEALER_XFS_HEALER_H_ */
healer/fsrepair.c +14 -2
index 907afca3dba8a7..4534104f8a6ac1 100644
@@ -164,7 +164,7 @@ try_repair_rtgroup(
 static void
 try_repair_inode(
 	struct healer_ctx			*ctx,
-	const struct hme_prefix			*pfx,
+	const struct hme_prefix			*orig_pfx,
 	int					mnt_fd,
 	const struct xfs_health_monitor_event	*hme)
 {
@@ -182,13 +182,25 @@ try_repair_inode(
 		{0,		0},
 	};
 #undef X
-	const struct u32_scrub *f;
+	struct hme_prefix	new_pfx;
+	const struct hme_prefix	*pfx = orig_pfx;
+	const struct u32_scrub	*f;
 
 	foreach_scrub_type(f, hme->e.inode.mask, INODE_STRUCTURES) {
 		enum repair_outcome	outcome =
 			xfs_repair_metadata(mnt_fd, f->scrub_type,
 					0, hme->e.inode.ino, hme->e.inode.gen);
 
+		/*
+		 * Try again to find the file path, maybe we fixed the dir
+		 * tree.
+		 */
+		if (!hme_prefix_has_path(pfx)) {
+			lookup_path(ctx, hme, &new_pfx);
+			if (hme_prefix_has_path(&new_pfx))
+				pfx = &new_pfx;
+		}
+
 		pthread_mutex_lock(&ctx->conlock);
 		report_health_repair(pfx, hme, f->event_mask, outcome);
 		pthread_mutex_unlock(&ctx->conlock);
healer/weakhandle.c +86 -0

95 lines not on this page · raw message

healer/xfs_healer.c +43 -2

64 lines not on this page · raw message

#14 lore raw

PATCH 13/26 xfs_healer: create a per-mount background monitoring service

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a systemd service definition for our self-healing filesystem daemon so that we can run it for every mounted filesystem. Add a hidden switch so that we can print the service unit name for fstests.

---
 healer/xfs_healer.h            |    1 
 healer/Makefile                |   22 ++++++++
 healer/system-xfs_healer.slice |   31 ++++++++++++
 healer/xfs_healer.c            |   16 ++++++
 healer/[email protected]  |  107 ++++++++++++++++++++++++++++++++++++++++
 5 files changed, 176 insertions(+), 1 deletion(-)
 create mode 100644 healer/system-xfs_healer.slice
 create mode 100644 healer/[email protected]
healer/xfs_healer.h +1 -0
index 6d12921245934c..679bdc95ae48f8 100644
@@ -26,6 +26,7 @@ struct healer_ctx {
 	int			everything;
 	int			foreground;
 	int			want_repair;
+	int			print_svcname;
 
 	/* fd and fs geometry for mount */
 	struct xfs_fd		mnt;
healer/Makefile +21 -1
index 981192b81af626..ee44aaee461250 100644
@@ -22,7 +22,23 @@ LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD)
 LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG)
 LLDFLAGS = -static
 
-default: depend $(LTCOMMAND)
[email protected]
+CFLAGS += -DXFS_HEALER_SVCNAME=\"$(XFS_HEALER_SVCNAME)\"
+
+ifeq ($(HAVE_SYSTEMD),yes)
+INSTALL_HEALER += install-systemd
+SYSTEMD_SERVICES=\
+	system-xfs_healer.slice \
+	$(XFS_HEALER_SVCNAME)
+OPTIONAL_TARGETS += $(SYSTEMD_SERVICES)
+endif
+
+default: depend $(LTCOMMAND) $(SYSTEMD_SERVICES)
+
+%.service: %.service.in $(builddefs)
+	@echo "    [SED]    $@"
+	$(Q)$(SED) -e "s|@pkg_libexec_dir@|$(PKG_LIBEXEC_DIR)|g" \
+		   < $< > $@
 
 include $(BUILDRULES)
 
@@ -32,6 +48,10 @@ install-healer: default
 	$(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR)
 	$(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR)
 
+install-systemd: default
+	$(INSTALL) -m 755 -d $(SYSTEMD_SYSTEM_UNIT_DIR)
+	$(INSTALL) -m 644 $(SYSTEMD_SERVICES) $(SYSTEMD_SYSTEM_UNIT_DIR)
+
 install-dev:
 
 -include .dep
healer/system-xfs_healer.slice new file +31 -0
new file mode 100644
index 00000000000000..b8f5bca03963ff
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+# Author: Darrick J. Wong <[email protected]>
+
+[Unit]
+Description=xfs_healer background service slice
+Before=slices.target
+
+[Slice]
+
+# If the CPU usage cgroup controller is available, don't use more than 2 cores
+# for all background processes.  One thread to read events, another to run
+# repairs.
+CPUQuota=200%
+CPUAccounting=true
+
+[Install]
+# As of systemd 249, the systemd cgroupv2 configuration code will drop resource
+# controllers from the root and system.slice cgroups at startup if it doesn't
+# find any direct dependencies that require a given controller.  Newly
+# activated units with resource control directives are created under the system
+# slice but do not cause a reconfiguration of the slice's resource controllers.
+# Hence we cannot put CPUQuota= into the xfs_healer service units directly.
+#
+# For the CPUQuota directive to have any effect, we must therefore create an
+# explicit definition file for the slice that systemd creates to contain the
+# xfs_healer instance units (e.g. [email protected]) and we must configure
+# this slice as a dependency of the system slice to establish the direct
+# dependency relation.
+WantedBy=system.slice
healer/xfs_healer.c +16 -0
index 63baf641cb6ec6..1a26ffe830e5fe 100644
@@ -407,6 +407,7 @@ enum long_opt_nr {
 	LOPT_HELP,
 	LOPT_QUIET,
 	LOPT_REPAIR,
+	LOPT_SVCNAME,
 
 	LOPT_MAX,
 };
@@ -438,6 +439,7 @@ main(
 		[LOPT_HELP]	   = {"help", no_argument, NULL, 0 },
 		[LOPT_QUIET]	   = {"quiet", no_argument, &ctx.log, 0 },
 		[LOPT_REPAIR]	   = {"repair", no_argument, &ctx.want_repair, 1 },
+		[LOPT_SVCNAME]	   = {"svcname", no_argument, &ctx.print_svcname, 1 },
 
 		[LOPT_MAX]	   = {NULL, 0, NULL, 0 },
 	};
@@ -474,6 +476,20 @@ main(
 
 	ctx.mntpoint = argv[optind];
 
+	if (ctx.print_svcname) {
+		char	unitname[PATH_MAX];
+
+		ret = systemd_path_instance_unit_name(XFS_HEALER_SVCNAME,
+				ctx.mntpoint, unitname, sizeof(unitname));
+		if (ret) {
+			perror(ctx.mntpoint);
+			return EXIT_FAILURE;
+		}
+
+		printf("%s\n", unitname);
+		return EXIT_SUCCESS;
+	}
+
 	ret = setup_monitor(&ctx);
 	if (ret)
 		goto out_events;
healer/[email protected] new file +107 -0

107 lines not on this page · raw message

#15 lore raw

PATCH 14/26 xfs_healer: create a service to start the per-mount healer service

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a daemon to wait for xfs mount events via fsnotify and start up the per-mount healer service. It's important that we're running in the same mount namespace as the mount, so we're a fanotify client to avoid having to filter the mount namespaces ourselves.

---
 libfrog/systemd.h                  |   23 +-
 configure.ac                       |    1 
 healer/Makefile                    |   16 +-
 healer/xfs_healer_start.c          |  368 ++++++++++++++++++++++++++++++++++++
 healer/xfs_healer_start.service.in |   85 ++++++++
 include/builddefs.in               |    5 
 m4/package_libcdev.m4              |   24 ++
 7 files changed, 511 insertions(+), 11 deletions(-)
 create mode 100644 healer/xfs_healer_start.c
 create mode 100644 healer/xfs_healer_start.service.in
libfrog/systemd.h +15 -8
index c96df4afa39aa6..8a0970282d1080 100644
@@ -22,6 +22,20 @@ static inline bool systemd_is_service(void)
 	return getenv("SERVICE_MODE") != NULL;
 }
 
+/* Special processing for a service/daemon program that is exiting. */
+static inline int
+systemd_service_exit_now(int ret)
+{
+	/*
+	 * If we're being run as a service, the return code must fit the LSB
+	 * init script action error guidelines, which is to say that we
+	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
+	 * section 22.2) and hope the admin will scan the log for what actually
+	 * happened.
+	 */
+	return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+}
+
 /* Special processing for a service/daemon program that is exiting. */
 static inline int
 systemd_service_exit(int ret)
@@ -35,14 +49,7 @@ systemd_service_exit(int ret)
 	 */
 	sleep(2);
 
-	/*
-	 * If we're being run as a service, the return code must fit the LSB
-	 * init script action error guidelines, which is to say that we
-	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
-	 * section 22.2) and hope the admin will scan the log for what actually
-	 * happened.
-	 */
-	return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+	return systemd_service_exit_now(ret);
 }
 
 #endif /* __LIBFROG_SYSTEMD_H__ */
configure.ac +1 -0
index 90af1f84035ee6..e098cf0530415b 100644
@@ -194,6 +194,7 @@ if test "$have_listmount" = "yes"; then
 	AC_HAVE_LISTMOUNT_NS_FD
 	AC_HAVE_STATMOUNT_SUPPORTED_MASK
 fi
+AC_HAVE_FANOTIFY_MOUNTINFO
 
 if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then
         AC_PACKAGE_CHECK_UBSAN
healer/Makefile +13 -3
index ee44aaee461250..1eeb727682008b 100644
@@ -9,6 +9,7 @@ include $(builddefs)
 INSTALL_HEALER = install-healer
 
 LTCOMMAND = xfs_healer
+BUILD_TARGETS = $(LTCOMMAND)
 
 CFILES = \
 fsrepair.c \
@@ -31,9 +32,18 @@ SYSTEMD_SERVICES=\
 	system-xfs_healer.slice \
 	$(XFS_HEALER_SVCNAME)
 OPTIONAL_TARGETS += $(SYSTEMD_SERVICES)
-endif
+endif # HAVE_SYSTEMD
 
-default: depend $(LTCOMMAND) $(SYSTEMD_SERVICES)
+ifeq ($(HAVE_HEALER_START_DEPS),yes)
+BUILD_TARGETS += xfs_healer_start
+SYSTEMD_SERVICES += xfs_healer_start.service
+endif # xfs_healer_start deps
+
+default: depend $(BUILD_TARGETS) $(SYSTEMD_SERVICES)
+
+xfs_healer_start: $(SUBDIRS) xfs_healer_start.o $(LTDEPENDENCIES)
+	@echo "    [LD]     $@"
+	$(Q)$(LTLINK) -o $@ $(LDFLAGS) xfs_healer_start.o $(LDLIBS)
 
 %.service: %.service.in $(builddefs)
 	@echo "    [SED]    $@"
@@ -46,7 +56,7 @@ install: $(INSTALL_HEALER)
 
 install-healer: default
 	$(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR)
-	$(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR)
+	$(INSTALL) -m 755 $(BUILD_TARGETS) $(PKG_LIBEXEC_DIR)
 
 install-systemd: default
 	$(INSTALL) -m 755 -d $(SYSTEMD_SYSTEM_UNIT_DIR)
healer/xfs_healer_start.c new file +368 -0

368 lines not on this page · raw message

healer/xfs_healer_start.service.in new file +85 -0

85 lines not on this page · raw message

include/builddefs.in +5 -0
index 0ab2bf1702f0f0..bdba9cd9037900 100644
@@ -124,6 +124,7 @@ HAVE_LISTMOUNT = @have_listmount@
 HAVE_LISTMOUNT_NS_FD = @have_listmount_ns_fd@
 HAVE_STATMOUNT_SUPPORTED_MASK = @have_statmount_supported_mask@
 NEED_INTERNAL_STATMOUNT = @need_internal_statmount@
+HAVE_FANOTIFY_MOUNTINFO = @have_fanotify_mountinfo@
 
 GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall
 #	   -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
@@ -159,6 +160,10 @@ ifeq ($(HAVE_LIBURCU_ATOMIC64),yes)
 PCFLAGS += -DHAVE_LIBURCU_ATOMIC64
 endif
 
+ifeq ($(ENABLE_HEALER)$(HAVE_SYSTEMD)$(HAVE_LISTMOUNT)$(HAVE_FANOTIFY_MOUNTINFO),yesyesyesyes)
+HAVE_HEALER_START_DEPS = yes
+endif
+
 SANITIZER_CFLAGS += @addrsan_cflags@ @threadsan_cflags@ @ubsan_cflags@ @autovar_init_cflags@
 SANITIZER_LDFLAGS += @addrsan_ldflags@ @threadsan_ldflags@ @ubsan_ldflags@
 
m4/package_libcdev.m4 +24 -0
index ec4a3ef444b705..9586bc01fe0f25 100644
@@ -452,3 +452,27 @@ AC_DEFUN([AC_HAVE_STATMOUNT_SUPPORTED_MASK],
     AC_SUBST(have_statmount_supported_mask)
     AC_SUBST(need_internal_statmount)
   ])
+
+#
+# Check if fanotify will give us mount notifications (6.15).
+#
+AC_DEFUN([AC_HAVE_FANOTIFY_MOUNTINFO],
+  [AC_MSG_CHECKING([for fanotify mount events])
+    AC_LINK_IFELSE(
+    [AC_LANG_PROGRAM([[
+#define _GNU_SOURCE
+#include <stdlib.h>
+#include <fcntl.h>
+#include <sys/fanotify.h>
+  ]], [[
+	struct fanotify_event_info_mnt info;
+
+	int fan_fd = fanotify_init(FAN_REPORT_MNT, 0);
+	fanotify_mark(fan_fd, FAN_MARK_ADD | FAN_MARK_MNTNS, FAN_MNT_ATTACH,
+			-1, NULL);
+  ]])
+    ], have_fanotify_mountinfo=yes
+       AC_MSG_RESULT(yes),
+       AC_MSG_RESULT(no))
+    AC_SUBST(have_fanotify_mountinfo)
+  ])
#16 lore raw

PATCH 15/26 xfs_healer: don't start service if kernel support unavailable

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use ExecCondition= in the system service to check if kernel support for the health monitor is available. If not, we don't want to run the service, have it fail, and generate a bunch of silly log messages.

---
 healer/xfs_healer.h           |    1 +
 healer/xfs_healer.c           |   47 ++++++++++++++++++++++++++++++++---------
 healer/[email protected] |    1 +
 3 files changed, 39 insertions(+), 10 deletions(-)
healer/xfs_healer.h +1 -0
index 679bdc95ae48f8..7caa6c66a59c6f 100644
@@ -27,6 +27,7 @@ struct healer_ctx {
 	int			foreground;
 	int			want_repair;
 	int			print_svcname;
+	int			support_check;
 
 	/* fd and fs geometry for mount */
 	struct xfs_fd		mnt;
healer/xfs_healer.c +37 -10

104 lines not on this page · raw message

healer/[email protected] +1 -0
index 385257872b0cbb..53f89cf9c4333d 100644
@@ -17,6 +17,7 @@ RequiresMountsFor=%f
 [Service]
 Type=exec
 Environment=SERVICE_MODE=1
+ExecCondition=@pkg_libexec_dir@/xfs_healer --supported %f
 ExecStart=@pkg_libexec_dir@/xfs_healer %f
 SyslogIdentifier=%N
 
#17 lore raw

PATCH 16/26 xfs_healer: use the autofsck fsproperty to select mode

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make the xfs_healer background service query the autofsck filesystem property to figure out which operating mode it should use.

---
 healer/xfs_healer.h    |    1 
 libfrog/fsproperties.h |    5 ++
 healer/xfs_healer.c    |  102 +++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 105 insertions(+), 3 deletions(-)
healer/xfs_healer.h +1 -0
index 7caa6c66a59c6f..a2a46053928e33 100644
@@ -28,6 +28,7 @@ struct healer_ctx {
 	int			want_repair;
 	int			print_svcname;
 	int			support_check;
+	int			autofsck;
 
 	/* fd and fs geometry for mount */
 	struct xfs_fd		mnt;
libfrog/fsproperties.h +5 -0
index 11d6530bc9a6d6..1cf90d058765b2 100644
@@ -52,6 +52,11 @@ bool fsprop_validate(const char *name, const char *value);
 
 #define FSPROP_AUTOFSCK_NAME		"autofsck"
 
+/* filesystem property name for fgetxattr */
+#define VFS_FSPROP_AUTOFSCK_NAME	(FSPROP_NAMESPACE \
+					 FSPROP_NAME_PREFIX \
+					 FSPROP_AUTOFSCK_NAME)
+
 enum fsprop_autofsck {
 	FSPROP_AUTOFSCK_UNSET = 0,	/* do not set property */
 	FSPROP_AUTOFSCK_NONE,		/* no background scrubs */
healer/xfs_healer.c +99 -3

161 lines not on this page · raw message

#18 lore raw

PATCH 17/26 xfs_healer: run full scrub after lost corruption events or targeted repair failure

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If we fail to perform a spot repair of metadata or the kernel tells us that it lost corruption events due to queue limits, initiate a full run of the online fsck service to try to fix the error.

---
 healer/xfs_healer.h  |    3 ++
 healer/Makefile      |    2 +
 healer/fsrepair.c    |   81 +++++++++++++++++++++++++++++++++++++++++++++-----
 healer/weakhandle.c  |   13 ++++++++
 healer/xfs_healer.c  |    7 ++++
 include/builddefs.in |    1 +
 scrub/Makefile       |    7 ++--
 7 files changed, 102 insertions(+), 12 deletions(-)
healer/xfs_healer.h +3 -0
index a2a46053928e33..e1370323bbd66a 100644
@@ -72,6 +72,7 @@ void lookup_path(struct healer_ctx *ctx,
 int repair_metadata(struct healer_ctx *ctx, const struct hme_prefix *pfx,
 		const struct xfs_health_monitor_event *hme);
 bool healer_can_repair(struct healer_ctx *ctx);
+void run_full_repair(struct healer_ctx *ctx);
 
 /* weakhandle.c */
 int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
@@ -80,5 +81,7 @@ int weakhandle_reopen(struct weakhandle *wh, int *fd);
 void weakhandle_free(struct weakhandle **whp);
 int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
 		char *path, size_t pathlen);
+int weakhandle_instance_unit_name(struct weakhandle *wh, const char *template,
+		char *unitname, size_t unitnamelen);
 
 #endif /* XFS_HEALER_XFS_HEALER_H_ */
healer/Makefile +2 -0
index 1eeb727682008b..b8ffce33e90d18 100644
@@ -19,6 +19,8 @@ xfs_healer.c
 HFILES = \
 xfs_healer.h
 
+CFLAGS+=-DXFS_SCRUB_SVCNAME=\"$(XFS_SCRUB_SVCNAME)\"
+
 LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD)
 LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG)
 LLDFLAGS = -static
healer/fsrepair.c +73 -8

156 lines not on this page · raw message

healer/weakhandle.c +13 -0
index 8950e0eb1e5a43..849aa2882700d4 100644
@@ -13,6 +13,7 @@
 #include "libfrog/workqueue.h"
 #include "libfrog/getparents.h"
 #include "libfrog/paths.h"
+#include "libfrog/systemd.h"
 #include "xfs_healer.h"
 
 struct weakhandle {
@@ -199,3 +200,15 @@ weakhandle_getpath_for(
 	close(mnt_fd);
 	return ret;
 }
+
+/* Compute the systemd instance unit name for this mountpoint. */
+int
+weakhandle_instance_unit_name(
+	struct weakhandle	*wh,
+	const char		*template,
+	char			*unitname,
+	size_t			unitnamelen)
+{
+	return systemd_path_instance_unit_name(template, wh->mntpoint,
+			unitname, unitnamelen);
+}
healer/xfs_healer.c +7 -0
index f4bee495979324..09b88c754a550c 100644
@@ -138,6 +138,13 @@ handle_event(
 		pthread_mutex_unlock(&ctx->conlock);
 	}
 
+	/*
+	 * If we didn't ask for all the metadata reports (including the healthy
+	 * ones) and the kernel tells us it lost something, run the full scan.
+	 */
+	if (hme->type == XFS_HEALTH_MONITOR_TYPE_LOST && !ctx->everything)
+		run_full_repair(ctx);
+
 	/* Initiate a repair if appropriate. */
 	if (will_repair)
 		repair_metadata(ctx, &pfx, hme);
include/builddefs.in +1 -0
index bdba9cd9037900..3b52d1afd7031c 100644
@@ -62,6 +62,7 @@ MKFS_CFG_DIR	= @datadir@/@pkg_name@/mkfs
 PKG_STATE_DIR	= @localstatedir@/lib/@pkg_name@
 
 XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP=$(PKG_STATE_DIR)/xfs_scrub_all_media.stamp
[email protected]
 
 CC		= @cc@
 BUILD_CC	= @BUILD_CC@
scrub/Makefile +3 -4
index ff79a265762332..aee49bfce100e2 100644
@@ -8,7 +8,6 @@ include $(builddefs)
 
 SCRUB_PREREQS=$(HAVE_GETFSMAP)
 
[email protected]
 [email protected]
 
 ifeq ($(SCRUB_PREREQS),yes)
@@ -21,7 +20,7 @@ XFS_SCRUB_SERVICE_ARGS = -b -o autofsck
 ifeq ($(HAVE_SYSTEMD),yes)
 INSTALL_SCRUB += install-systemd
 SYSTEMD_SERVICES=\
-	$(scrub_svcname) \
+	$(XFS_SCRUB_SVCNAME) \
 	[email protected] \
 	$(scrub_media_svcname) \
 	[email protected] \
@@ -123,7 +122,7 @@ xfs_scrub_all.timer: xfs_scrub_all.timer.in $(builddefs)
 $(XFS_SCRUB_ALL_PROG): $(XFS_SCRUB_ALL_PROG).in $(builddefs) $(TOPDIR)/libfrog/gettext.py
 	@echo "    [SED]    $@"
 	$(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \
-		   -e "s|@scrub_svcname@|$(scrub_svcname)|g" \
+		   -e "s|@scrub_svcname@|$(XFS_SCRUB_SVCNAME)|g" \
 		   -e "s|@scrub_media_svcname@|$(scrub_media_svcname)|g" \
 		   -e "s|@pkg_version@|$(PKG_VERSION)|g" \
 		   -e "s|@stampfile@|$(XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP)|g" \
@@ -137,7 +136,7 @@ $(XFS_SCRUB_ALL_PROG): $(XFS_SCRUB_ALL_PROG).in $(builddefs) $(TOPDIR)/libfrog/g
 xfs_scrub_fail: xfs_scrub_fail.in $(builddefs)
 	@echo "    [SED]    $@"
 	$(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \
-		   -e "s|@scrub_svcname@|$(scrub_svcname)|g" \
+		   -e "s|@scrub_svcname@|$(XFS_SCRUB_SVCNAME)|g" \
 		   -e "s|@pkg_version@|$(PKG_VERSION)|g"  < $< > $@
 	$(Q)chmod a+x $@
 
#19 lore raw

PATCH 18/26 xfs_healer: use getmntent to find moved filesystems

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

It's possible that a mounted filesystem can move mountpoints between the time of the initial mount (at which point xfs_healer starts) and when it actually wants to start a repair. When this happens, weakhandle::mountpoint becomes obsolete and opening it will either fail with ENOENT or the handle revalidation will return ESTALE.

However, we do still have a means to find the mounted filesystem -- the fsname parameter (aka the path to the data device at mount time). This is record in /proc/mounts, which means that we can iterate getmntent to see if we can find the mount elsewhere.

As documented a few patches ago, this would be easier if we had revocable fds that didn't pin mounts, but that's a very huge ask.

This getmntent code enables xfs_healer to find a filesystem that has been bind mounted in a new place and the original mountpoint detached:

# mount /dev/sda /mnt
# xfs_healer /mnt &
# mount /mnt /opt --bind
# umount /mnt

The key here is that each bind mount gets a separate struct mount object.

---
 healer/weakhandle.c |   55 +++++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 51 insertions(+), 4 deletions(-)
healer/weakhandle.c +51 -4
index 849aa2882700d4..5df5207514e38e 100644
@@ -65,10 +65,14 @@ weakhandle_alloc(
 	return -1;
 }
 
-/* Reopen a file handle obtained via weak reference. */
-int
-weakhandle_reopen(
+/*
+ * Reopen a file handle obtained via weak reference, using the given path to a
+ * mount point.
+ */
+static int
+weakhandle_reopen_from(
 	struct weakhandle	*wh,
+	const char		*path,
 	int			*fd)
 {
 	void			*hanp;
@@ -78,7 +82,7 @@ weakhandle_reopen(
 
 	*fd = -1;
 
-	mnt_fd = open(wh->mntpoint, O_RDONLY);
+	mnt_fd = open(path, O_RDONLY);
 	if (mnt_fd < 0)
 		return -1;
 
@@ -102,6 +106,49 @@ weakhandle_reopen(
 	return -1;
 }
 
+/* Reopen a file handle obtained via weak reference. */
+int
+weakhandle_reopen(
+	struct weakhandle	*wh,
+	int			*fd)
+{
+	FILE			*mtab;
+	struct mntent		*mnt;
+	int			ret;
+
+	/* First try reopening using the original mountpoint */
+	ret = weakhandle_reopen_from(wh, wh->mntpoint, fd);
+	if (!ret)
+		return 0;
+
+	/*
+	 * That didn't work, so now walk /proc/mounts to find a mount with the
+	 * same fsname (aka xfs data device path) as when we started.
+	 */
+	mtab = setmntent(_PATH_PROC_MOUNTS, "r");
+	if (!mtab)
+		return -1;
+
+	while ((mnt = getmntent(mtab)) != NULL) {
+		if (strcmp(mnt->mnt_type, "xfs"))
+			continue;
+		if (strcmp(mnt->mnt_fsname, wh->fsname))
+			continue;
+
+		ret = weakhandle_reopen_from(wh, mnt->mnt_dir, fd);
+		if (!ret)
+			break;
+	}
+
+	if (*fd < 0) {
+		errno = ESTALE;
+		ret = -1;
+	}
+
+	endmntent(mtab);
+	return ret;
+}
+
 /* Tear down a weak handle */
 void
 weakhandle_free(
#20 lore raw

PATCH 19/26 xfs_healer: use statmount to find moved filesystems even faster

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

As noted in the previous patch, it's possible that a mounted filesystem can move mountpoints between the time of the initial mount (at which point xfs_healer starts) and when it actually wants to start a repair. The previous patch fixed that problem by using getmntent to walk /proc/self/mounts to see if it finds a mount with the same "source" name, aka data device.

However, this is really slow if there are a lot of filesystems because we end up wading through a lot of irrelevant information. However, statmount() can help us here because as of Linux 7.0 we can open the passed-in path at startup, call statmount() on it to retrieve the mnt_id, and then call it again later with that same mnt_id to find the mountpoint. Luckily xfs_healthmon didn't get merged until 7.0 so it's more or less guaranteed to be there if XFS_IOC_HEALTH_MONITOR succeeds.

Obviously if this doesn't work, we can fall back to the slow walk.

This statmount code enables xfs_healer to find a filesystem that has had its mountpoint moved to a different place in the directory tree without the use of bind mounts and without needing to walk the entire mount list:

# mount -t tmpfs urk /mnt
# mount --make-rprivate /mnt
# mkdir -p /mnt/a /mnt/b
# mount /dev/sda /mnt/a
# mount --move /mnt/a /mnt/b

The key here is that the struct mount object is moved, and no new ones are created. Therefore, the original mnt_id is still usable.

---
 healer/xfs_healer.h |    7 +++++--
 healer/weakhandle.c |   24 ++++++++++++++++++++++++
 healer/xfs_healer.c |   37 +++++++++++++++++++++++++++++++++++--
 3 files changed, 64 insertions(+), 4 deletions(-)
healer/xfs_healer.h +5 -2
index e1370323bbd66a..96e146f266629a 100644
@@ -39,6 +39,9 @@ struct healer_ctx {
 	/* Shared reference to the getmntent fsname for reconnecting */
 	const char		*fsname;
 
+	/* Mount id for faster reconnecting */
+	uint64_t		mnt_id;
+
 	/* weak file handle so we can reattach to filesystem */
 	struct weakhandle	*wh;
 
@@ -75,8 +78,8 @@ bool healer_can_repair(struct healer_ctx *ctx);
 void run_full_repair(struct healer_ctx *ctx);
 
 /* weakhandle.c */
-int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
-		struct weakhandle **whp);
+int weakhandle_alloc(int fd, const char *mountpoint, uint64_t mnt_id,
+		const char *fsname, struct weakhandle **whp);
 int weakhandle_reopen(struct weakhandle *wh, int *fd);
 void weakhandle_free(struct weakhandle **whp);
 int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
healer/weakhandle.c +24 -0
index 5df5207514e38e..358c553f883f3d 100644
@@ -14,6 +14,7 @@
 #include "libfrog/getparents.h"
 #include "libfrog/paths.h"
 #include "libfrog/systemd.h"
+#include "libfrog/statmount.h"
 #include "xfs_healer.h"
 
 struct weakhandle {
@@ -23,6 +24,9 @@ struct weakhandle {
 	/* Shared reference to the getmntent fsname for reconnecting */
 	const char		*fsname;
 
+	/* Mount id for faster reconnecting */
+	uint64_t		mnt_id;
+
 	/* handle to root dir */
 	void			*hanp;
 	size_t			hlen;
@@ -33,6 +37,7 @@ int
 weakhandle_alloc(
 	int			fd,
 	const char		*mountpoint,
+	uint64_t		mnt_id,
 	const char		*fsname,
 	struct weakhandle	**whp)
 {
@@ -51,6 +56,7 @@ weakhandle_alloc(
 		return -1;
 
 	wh->mntpoint = mountpoint;
+	wh->mnt_id = mnt_id;
 	wh->fsname = fsname;
 
 	ret = fd_to_handle(fd, &wh->hanp, &wh->hlen);
@@ -112,6 +118,9 @@ weakhandle_reopen(
 	struct weakhandle	*wh,
 	int			*fd)
 {
+	const size_t		smbuf_size =
+		libfrog_statmount_sizeof(PATH_MAX);
+	struct statmount	*smbuf = alloca(smbuf_size);
 	FILE			*mtab;
 	struct mntent		*mnt;
 	int			ret;
@@ -121,6 +130,21 @@ weakhandle_reopen(
 	if (!ret)
 		return 0;
 
+	/*
+	 * The original mountpoint didn't work, which means the mount might
+	 * have been moved.  Look up the mountpoint for the mount id that we
+	 * captured earlier, which is a quick lookup if there are many mounts.
+	 * Note that @ret is nonzero here.
+	 */
+	ret = libfrog_statmount(wh->mnt_id, DEFAULT_MOUNTNS_FD,
+			STATMOUNT_MNT_POINT, smbuf, smbuf_size);
+	if (ret || !(smbuf->mask & STATMOUNT_MNT_POINT))
+		goto fallback;
+	ret = weakhandle_reopen_from(wh, smbuf->str + smbuf->mnt_point, fd);
+	if (!ret)
+		return 0;
+
+fallback:
 	/*
 	 * That didn't work, so now walk /proc/mounts to find a mount with the
 	 * same fsname (aka xfs data device path) as when we started.
healer/xfs_healer.c +35 -2
index 09b88c754a550c..b91d7f16774b75 100644
@@ -15,6 +15,7 @@
 #include "libfrog/workqueue.h"
 #include "libfrog/systemd.h"
 #include "libfrog/fsproperties.h"
+#include "libfrog/statmount.h"
 #include "xfs_healer.h"
 
 /* Program name; needed for libfrog error reports. */
@@ -163,11 +164,43 @@ try_capture_fsinfo(
 {
 	struct mntent		*mnt;
 	FILE			*mtp;
-	char			rpath[PATH_MAX], rmnt_dir[PATH_MAX];
+	const size_t		smbuf_size =
+		libfrog_statmount_sizeof(PATH_MAX + 128);
+	struct statmount	*smbuf = alloca(smbuf_size);
+	char			*rmnt_dir = smbuf->str;
+	char			rpath[PATH_MAX];
+	int			ret;
 
 	if (!realpath(ctx->mntpoint, rpath))
 		return -1;
 
+	/*
+	 * In Linux 7.0 we can do statmount on an open file, which means that
+	 * we can capture the mnt_id, mount point, and fsname, which can help
+	 * us find a mount --move'd elsewhere in the directory tree.
+	 */
+	ret = libfrog_fstatmount(ctx->mnt.fd, STATMOUNT_MNT_POINT, smbuf,
+			smbuf_size);
+	if (ret || !(smbuf->mask & STATMOUNT_MNT_POINT))
+		goto fallback;
+	if (strcmp(rpath, smbuf->str + smbuf->mnt_point))
+		goto fallback;
+
+	ret = libfrog_fstatmount(ctx->mnt.fd,
+			STATMOUNT_SB_SOURCE | STATMOUNT_MNT_BASIC,
+			smbuf, smbuf_size);
+	if (ret || !(smbuf->mask & STATMOUNT_SB_SOURCE))
+		goto fallback;
+
+	ctx->fsname = strdup(smbuf->str + smbuf->sb_source);
+	ctx->mnt_id = smbuf->mnt_id;
+	return 0;
+
+fallback:
+	/*
+	 * If statmount isn't available for whatever reason, fall back to
+	 * walking the mount table via getmntent.
+	 */
 	mtp = setmntent(_PATH_PROC_MOUNTS, "r");
 	if (mtp == NULL)
 		return -1;
@@ -341,7 +374,7 @@ setup_monitor(
 	 * paths for logging.
 	 */
 	if (ctx->want_repair || healer_has_parent(ctx)) {
-		ret = weakhandle_alloc(ctx->mnt.fd, ctx->mntpoint,
+		ret = weakhandle_alloc(ctx->mnt.fd, ctx->mntpoint, ctx->mnt_id,
 				ctx->fsname, &ctx->wh);
 		if (ret) {
 			fprintf(stderr, "%s: %s: %s\n", ctx->mntpoint,
#22 lore raw

PATCH 20/26 xfs_healer: validate that repair fds point to the monitored fs

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When xfs_healer reopens a mountpoint to perform a repair, it should validate that the opened fd points to a file on the same filesystem as the one being monitored.

---
 healer/xfs_healer.h |    4 +++-
 healer/fsrepair.c   |   18 +++++++++++++++++-
 healer/weakhandle.c |   23 +++++++++++++++++------
 3 files changed, 37 insertions(+), 8 deletions(-)
healer/xfs_healer.h +3 -1
index 96e146f266629a..c6692375dda6bf 100644
@@ -80,7 +80,9 @@ void run_full_repair(struct healer_ctx *ctx);
 /* weakhandle.c */
 int weakhandle_alloc(int fd, const char *mountpoint, uint64_t mnt_id,
 		const char *fsname, struct weakhandle **whp);
-int weakhandle_reopen(struct weakhandle *wh, int *fd);
+typedef bool (*weakhandle_fd_t)(int mnt_fd, void *data);
+int weakhandle_reopen(struct weakhandle *wh, int *fd,
+		weakhandle_fd_t is_acceptable, void *data);
 void weakhandle_free(struct weakhandle **whp);
 int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
 		char *path, size_t pathlen);
healer/fsrepair.c +17 -1
index 9f8c128e395ebc..002e5e78fcf22e 100644
@@ -233,6 +233,22 @@ try_repair_inode(
 	return REPAIR_DONE;
 }
 
+/* Make sure the reopened file is on the same fs as the monitor. */
+static bool
+is_same_fs(
+	int				mnt_fd,
+	void				*data)
+{
+	struct xfs_health_file_on_monitored_fs hms = {
+		.fd = mnt_fd,
+	};
+	FILE				*mon_fp = data;
+	int				ret;
+
+	ret = ioctl(fileno(mon_fp), XFS_IOC_HEALTH_FD_ON_MONITORED_FS, &hms);
+	return ret == 0;
+}
+
 /* Repair a metadata corruption. */
 int
 repair_metadata(
@@ -244,7 +260,7 @@ repair_metadata(
 	int					repair_fd;
 	int					ret;
 
-	ret = weakhandle_reopen(ctx->wh, &repair_fd);
+	ret = weakhandle_reopen(ctx->wh, &repair_fd, is_same_fs, ctx->mon_fp);
 	if (ret) {
 		fprintf(stderr, "%s: %s: %s\n", ctx->mntpoint,
 				_("cannot open filesystem to repair"),
healer/weakhandle.c +17 -6
index 358c553f883f3d..7b8cef0a63f971 100644
@@ -79,7 +79,9 @@ static int
 weakhandle_reopen_from(
 	struct weakhandle	*wh,
 	const char		*path,
-	int			*fd)
+	int			*fd,
+	weakhandle_fd_t		is_acceptable,
+	void			*data)
 {
 	void			*hanp;
 	size_t			hlen;
@@ -101,6 +103,11 @@ weakhandle_reopen_from(
 		goto out_handle;
 	}
 
+	if (is_acceptable && !is_acceptable(mnt_fd, data)) {
+		errno = ESTALE;
+		goto out_handle;
+	}
+
 	free_handle(hanp, hlen);
 	*fd = mnt_fd;
 	return 0;
@@ -116,7 +123,9 @@ weakhandle_reopen_from(
 int
 weakhandle_reopen(
 	struct weakhandle	*wh,
-	int			*fd)
+	int			*fd,
+	weakhandle_fd_t		is_acceptable,
+	void			*data)
 {
 	const size_t		smbuf_size =
 		libfrog_statmount_sizeof(PATH_MAX);
@@ -126,7 +135,7 @@ weakhandle_reopen(
 	int			ret;
 
 	/* First try reopening using the original mountpoint */
-	ret = weakhandle_reopen_from(wh, wh->mntpoint, fd);
+	ret = weakhandle_reopen_from(wh, wh->mntpoint, fd, is_acceptable, data);
 	if (!ret)
 		return 0;
 
@@ -140,7 +149,8 @@ weakhandle_reopen(
 			STATMOUNT_MNT_POINT, smbuf, smbuf_size);
 	if (ret || !(smbuf->mask & STATMOUNT_MNT_POINT))
 		goto fallback;
-	ret = weakhandle_reopen_from(wh, smbuf->str + smbuf->mnt_point, fd);
+	ret = weakhandle_reopen_from(wh, smbuf->str + smbuf->mnt_point, fd,
+			is_acceptable, data);
 	if (!ret)
 		return 0;
 
@@ -159,7 +169,8 @@ weakhandle_reopen(
 		if (strcmp(mnt->mnt_fsname, wh->fsname))
 			continue;
 
-		ret = weakhandle_reopen_from(wh, mnt->mnt_dir, fd);
+		ret = weakhandle_reopen_from(wh, mnt->mnt_dir, fd,
+				is_acceptable, data);
 		if (!ret)
 			break;
 	}
@@ -244,7 +255,7 @@ weakhandle_getpath_for(
 	fakehandle.ha_fid.fid_ino = ino;
 	fakehandle.ha_fid.fid_gen = gen;
 
-	ret = weakhandle_reopen(wh, &mnt_fd);
+	ret = weakhandle_reopen(wh, &mnt_fd, NULL, NULL);
 	if (ret)
 		return ret;
 
#23 lore raw

PATCH 21/26 xfs_healer: add a manual page

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a new section 8 manpage for this service daemon so others can read about what this program is supposed to do.

---
 man/man8/Makefile           |   40 +++++++++++++---
 man/man8/xfs_healer.8       |  109 +++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_healer_start.8 |   37 +++++++++++++++
 3 files changed, 180 insertions(+), 6 deletions(-)
 create mode 100644 man/man8/xfs_healer.8
 create mode 100644 man/man8/xfs_healer_start.8
man/man8/Makefile +34 -6
index 5be76ab727a1fe..05710f85ae89ad 100644
@@ -7,13 +7,41 @@ include $(TOPDIR)/include/builddefs
 
 MAN_SECTION	= 8
 
-ifneq ("$(ENABLE_SCRUB)","yes")
-  MAN_PAGES = $(filter-out xfs_scrub%,$(shell echo *.$(MAN_SECTION)))
-else
-  MAN_PAGES = $(shell echo *.$(MAN_SECTION))
-  MAN_PAGES += xfs_scrub_all.8
+MAN_PAGES = \
+	fsck.xfs.8 \
+	mkfs.xfs.8 \
+	xfs_admin.8 \
+	xfs_bmap.8 \
+	xfs_copy.8 \
+	xfs_db.8 \
+	xfs_estimate.8 \
+	xfs_freeze.8 \
+	xfs_fsr.8 \
+	xfs_growfs.8 \
+	xfs_info.8 \
+	xfs_io.8 \
+	xfs_logprint.8 \
+	xfs_mdrestore.8 \
+	xfs_metadump.8 \
+	xfs_mkfile.8 \
+	xfs_ncheck.8 \
+	xfs_property.8 \
+	xfs_protofile.8 \
+	xfs_quota.8 \
+	xfs_repair.8 \
+	xfs_rtcp.8 \
+	xfs_spaceman.8
+
+ifeq ($(ENABLE_HEALER),yes)
+  MAN_PAGES += xfs_healer.8
 endif
-MAN_PAGES	+= mkfs.xfs.8
+ifeq ($(HAVE_HEALER_START_DEPS),yes)
+  MAN_PAGES += xfs_healer_start.8
+endif
+ifeq ($(ENABLE_SCRUB),yes)
+  MAN_PAGES += xfs_scrub.8 xfs_scrub_all.8
+endif
+
 MAN_DEST	= $(PKG_MAN_DIR)/man$(MAN_SECTION)
 LSRCFILES	= $(MAN_PAGES)
 DIRT		= mkfs.xfs.8 xfs_scrub_all.8
man/man8/xfs_healer.8 new file +109 -0

109 lines not on this page · raw message

man/man8/xfs_healer_start.8 new file +37 -0
new file mode 100644
index 00000000000000..9e424432a513fe
@@ -0,0 +1,37 @@
+.TH xfs_healer_start 8
+.SH NAME
+xfs_healer_start \- starts xfs_healer instances
+.SH SYNOPSIS
+.B xfs_healer_start
+[
+.B OPTIONS
+]
+.br
+.B xfs_healer \-V
+.SH DESCRIPTION
+.B xfs_healer_start
+starts the xfs_healer service whenever the kernel mounts an XFS filesystem in
+the current mount namespace.
+.PP
+.B WARNING!
+This program is
+.BR EXPERIMENTAL ","
+which means that its behavior and interface
+could change at any time!
+
+Normally this program runs as a systemd service.
+
+.SH OPTIONS
+.TP
+.B \-\-supported
+Check if the kernel supports listening for mount events.
+Exits with 0 if it does, and non-zero if not.
+.TP
+.BI "\-\-mountns " path
+Monitor the given mount namespace.
+Defaults to the mount namespace associated with the process itself.
+.TP
+.BI \-V
+Prints the version number and exit.
+.SH SEE ALSO
+.BR xfs_healer (8).
#24 lore raw

PATCH 22/26 xfs_scrub: print systemd service names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a hidden switch to xfs_scrub to emit systemd service names for XFS services targetting filesystems paths instead of opencoding the computation in things like fstests.

---
 scrub/xfs_scrub.h |    3 +++
 scrub/Makefile    |    7 +++++--
 scrub/xfs_scrub.c |   50 ++++++++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 54 insertions(+), 6 deletions(-)
scrub/xfs_scrub.h +3 -0
index 6ee359f4cebd47..041c0fadaa93c0 100644
@@ -108,6 +108,9 @@ struct scrub_ctx {
 	 * this much space per volume.
 	 */
 	double			fstrim_block_pct;
+
+	/* CLI options, must be int */
+	int			print_svcname;
 };
 
 /*
scrub/Makefile +5 -2
index aee49bfce100e2..4aa0a7d836c342 100644
@@ -8,9 +8,12 @@ include $(builddefs)
 
 SCRUB_PREREQS=$(HAVE_GETFSMAP)
 
[email protected]
[email protected]
 
 ifeq ($(SCRUB_PREREQS),yes)
+CFLAGS+=-DXFS_SCRUB_SVCNAME=\"$(XFS_SCRUB_SVCNAME)\"
+CFLAGS+=-DXFS_SCRUB_MEDIA_SVCNAME=\"$(XFS_SCRUB_MEDIA_SVCNAME)\"
+
 LTCOMMAND = xfs_scrub
 INSTALL_SCRUB = install-scrub
 XFS_SCRUB_ALL_PROG = xfs_scrub_all.py
@@ -22,7 +25,7 @@ INSTALL_SCRUB += install-systemd
 SYSTEMD_SERVICES=\
 	$(XFS_SCRUB_SVCNAME) \
 	[email protected] \
-	$(scrub_media_svcname) \
+	$(XFS_SCRUB_MEDIA_SVCNAME) \
 	[email protected] \
 	xfs_scrub_all.service \
 	xfs_scrub_all_fail.service \
scrub/xfs_scrub.c +46 -4

80 lines not on this page · raw message

#25 lore raw

PATCH 23/26 xfs_io: add listmount and statmount commands

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add two new commands: one to list all mounts via statmount, now that we use this in xfs_healer_start, and another to statmount each open file.

---
 io/io.h           |    6 +
 io/Makefile       |    5 +
 io/init.c         |    1 
 io/listmount.c    |  361 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_io.8 |   66 ++++++++++
 5 files changed, 439 insertions(+)
 create mode 100644 io/listmount.c
io/io.h +6 -0
index 0f12b3cfed5e76..5f1f278d14a033 100644
@@ -164,3 +164,9 @@ void			fsprops_init(void);
 void			aginfo_init(void);
 void			healthmon_init(void);
 void			verifymedia_init(void);
+
+#ifdef HAVE_LISTMOUNT
+void			listmount_init(void);
+#else
+# define		listmount_init()	do { } while (0)
+#endif
io/Makefile +5 -0
index 79d5e172b8f31f..e25742b635396e 100644
@@ -90,6 +90,11 @@ ifeq ($(HAVE_GETFSMAP),yes)
 CFILES += fsmap.c
 endif
 
+ifeq ($(HAVE_LISTMOUNT),yes)
+CFILES += listmount.c
+LCFLAGS += -DHAVE_LISTMOUNT
+endif
+
 default: depend $(LTCOMMAND)
 
 include $(BUILDRULES)
io/init.c +1 -0
index f2a551ef559200..ba60cb2199639b 100644
@@ -94,6 +94,7 @@ init_commands(void)
 	fsprops_init();
 	healthmon_init();
 	verifymedia_init();
+	listmount_init();
 }
 
 /*
io/listmount.c new file +361 -0

361 lines not on this page · raw message

man/man8/xfs_io.8 +66 -0

72 lines not on this page · raw message

#26 lore raw

PATCH 24/26 mkfs: enable online repair if all backrefs are enabled

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If all backreferences are enabled in the filesystem, then enable online repair by default if the user didn't supply any other autofsck setting. Users might as well get full self-repair capability if they're paying for the extra metadata.

Note that it's up to each distro to enable the systemd services according to their own service activation policies. Debian policy is to enable all systemd services at package installation but they don't enable online fsck in their Kconfig so the services won't activate. RHEL and SUSE policy requires sysadmins to enable them explicitly unless the OS vendor also ships a systemd preset file enabling the services. Distros without systemd won't get any of the systemd services, obviously.

---
 mkfs/xfs_mkfs.c |    9 +++++++++
 1 file changed, 9 insertions(+)
mkfs/xfs_mkfs.c +9 -0
index 527a662f3ac858..f859626afdda36 100644
@@ -6296,6 +6296,15 @@ main(
 	if (mp->m_sb.sb_agcount > 1)
 		rewrite_secondary_superblocks(mp);
 
+	/*
+	 * If the filesystem has full backreferences and the user didn't
+	 * express an autofsck preference, enable online repair because they
+	 * might as well get some useful functionality from the extra metadata.
+	 */
+	if (cli.autofsck == FSPROP_AUTOFSCK_UNSET &&
+	    cli.sb_feat.rmapbt && cli.sb_feat.parent_pointers)
+		cli.autofsck = FSPROP_AUTOFSCK_REPAIR;
+
 	if (cli.autofsck != FSPROP_AUTOFSCK_UNSET)
 		set_autofsck(mp, &cli);
 
#27 lore raw

PATCH 25/26 debian/control: listify the build dependencies

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

This will make it less gross to add more build deps later.

---
 debian/control |   14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
debian/control +13 -1
index 6473c10be7f7d6..d50960fba205bb 100644
@@ -3,7 +3,19 @@ Section: admin
 Priority: optional
 Maintainer: XFS Development Team <[email protected]>
 Uploaders: Nathan Scott <[email protected]>, Anibal Monsalve Salazar <[email protected]>
-Build-Depends: libinih-dev (>= 53), uuid-dev, debhelper (>= 12), gettext, libtool, libedit-dev, libblkid-dev (>= 2.17), linux-libc-dev, libdevmapper-dev, libicu-dev, pkg-config, liburcu-dev, systemd-dev | systemd (<< 253-2~)
+Build-Depends: debhelper (>= 12),
+ gettext,
+ libblkid-dev (>= 2.17),
+ libdevmapper-dev,
+ libedit-dev,
+ libicu-dev,
+ libinih-dev (>= 53),
+ libtool,
+ liburcu-dev,
+ linux-libc-dev,
+ pkg-config,
+ systemd-dev | systemd (<< 253-2~),
+ uuid-dev
 Standards-Version: 4.0.0
 Homepage: https://xfs.wiki.kernel.org/
 
#28 lore raw

PATCH 26/26 debian: enable xfs_healer on the root filesystem by default

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that we're finished building autonomous repair, enable the healer service on the root filesystem by default. The root filesystem is mounted by the initrd prior to starting systemd, which is why the xfs_healer_start service cannot autostart the service for the root filesystem.

dh_installsystemd won't activate a template service (aka one with an at-sign in the name) even if it provides a DefaultInstance directive to make that possible. Hence we enable this explicitly via the postinst script.

Note that Debian enables services by default upon package installation, so this is consistent with their policies. Their kernel doesn't enable online fsck, so healer won't do much more than monitor for corruptions and log them.

---
 debian/postinst |    8 ++++++++
 debian/prerm    |   13 +++++++++++++
 debian/rules    |    3 ++-
 3 files changed, 23 insertions(+), 1 deletion(-)
 create mode 100644 debian/prerm
debian/postinst +8 -0
index d11c8d94a3cbe4..966dbb7626cab3 100644
@@ -21,5 +21,13 @@ case "${1}" in
 esac
 
 #DEBHELPER#
+#
+# dh_installsystemd doesn't handle template services even if we supply a
+# default instance, so we'll install it here.
+if [ -z "${DPKG_ROOT:-}" ] && [ -d /run/systemd/system ] ; then
+	if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
+		/bin/systemctl enable [email protected] || true
+	fi
+fi
 
 exit 0
debian/prerm new file +13 -0
new file mode 100644
index 00000000000000..c526dcdd1d7103
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -e
+
+# dh_installsystemd doesn't handle template services even if we supply a
+# default instance, so we'll install it here.
+if [ -z "${DPKG_ROOT:-}" ] && [ "$1" = remove ] && [ -d /run/systemd/system ] ; then
+	/bin/systemctl disable [email protected] || true
+fi
+
+#DEBHELPER#
+
+exit 0
debian/rules +2 -1
index 7c9f90e6c483ff..aaf99a95ce3df5 100755
@@ -97,4 +97,5 @@ override_dh_installdocs:
 	dh_installdocs -XCHANGES
 
 override_dh_installsystemd:
-	dh_installsystemd -p xfsprogs --no-restart-after-upgrade --no-stop-on-upgrade system-xfs_scrub.slice xfs_scrub_all.timer
+	dh_installsystemd -p xfsprogs --no-restart-after-upgrade --no-stop-on-upgrade system-xfs_scrub.slice xfs_scrub_all.timer system-xfs_healer.slice
+	dh_installsystemd -p xfsprogs --restart-after-upgrade xfs_healer_start.service
#29 lore raw

PATCH v10 2/2 xfs_scrub: refactor to XFS_IOC_VERIFY_MEDIA

In reply to Darrick J. Wong

Hi all,

Originally, this was a single patch inside the autonomous self healing patchset. However, various issues brought up during review of the v9 codebase made it obvious that more cleanups were necessary prior to shifting xfs_scrub to use the new media verification ioctl. Therefore, the xfs_scrub changes have been broken out into a separate patchset to contain the refactorings and new functionality.

v10: decouple the spacemap iterator and the read-verify pool code,
simplify the read-verify apis, use a fixed limit of 8 verify IO threads per device, improve the log media error reporting

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-media-verify-ioctl
---
Commits in this patchset:
* libfrog: allow bitmap_free to handle a null bitmap pointer
* mkfs: rename byte unit conversion macros
* libfrog: lift *BYTES helpers to convert.h
* xfs_scrub: report truncated devices as media errors
* xfs_scrub: fix i18n of the decode_special_owner return value
* scrub: remove the unused io_disk field in struct read_verify
* xfs_scrub: move read verification scheduling to phase6.c
* scrub: simplify the read_verify_pool_alloc interface
* xfs_scrub: don't pass the io_end_arg around everywhere
* scrub: use enum xfs_device for read verification
* xfs_scrub: rename nr_io_threads
* scrub: simplify verifier threads calculation
* xfs_scrub: move disk media verification error injection
* xfs_scrub: use the verify media ioctl during phase 6 if possible
* scrub: don't allocate disk for ioctl-based media verify
* xfs_scrub: perform media scanning of the log region
* xfs_scrub: index read-verify pools by xfs_device ids
* xfs_scrub: move failmap and other outputs into read_verify_pool
* xfs_scrub: clean up device-related error messages
* xfs_scrub: drop SCSI_VERIFY code from disk.
* xfs_scrub: raise media verification IO limits
* xfs_scrub: allow overrides of the media verification IO limits

---
 libfrog/convert.h         |    7 +
 scrub/disk.h              |    2 
 scrub/read_verify.h       |   34 ++-
 scrub/spacemap.h          |    5 
 scrub/xfs_scrub.h         |    9 -
 db/bmap_inflate.c         |    2 
 doc/README-env-vars.txt   |    4 
 libfrog/bitmap.c          |    3 
 libfrog/convert.c         |    7 -
 mdrestore/xfs_mdrestore.c |    5 
 mkfs/xfs_mkfs.c           |   38 ++--
 repair/agbtree.c          |    3 
 repair/bmap_repair.c      |    3 
 repair/rmap.c             |    3 
 repair/xfs_repair.c       |    5 
 scrub/common.c            |   19 +-
 scrub/disk.c              |  204 -------------------
 scrub/phase1.c            |   91 ++++++--
 scrub/phase5.c            |    6 -
 scrub/phase6.c            |  417 +++++++++++++++++++--------------------
 scrub/phase8.c            |    3 
 scrub/read_verify.c       |  485 +++++++++++++++++++++++++++++++++------------
 scrub/xfs_scrub.c         |    4 
 23 files changed, 722 insertions(+), 637 deletions(-)
#30 lore raw

PATCH 1/22 libfrog: allow bitmap_free to handle a null bitmap pointer

From: Darrick J. Wong <[email protected]>

Allow bitmap_free() callers to pass a pointer to a NULL pointer. This will help subsequent refactorings in xfs_scrub have cleaner bitmap_free callsites.

---
 libfrog/bitmap.c |    3 +++
 repair/rmap.c    |    3 +--
 scrub/phase5.c   |    6 ++----
 3 files changed, 6 insertions(+), 6 deletions(-)
libfrog/bitmap.c +3 -0
index 0308886d446ff2..6a3d852c25a46c 100644
@@ -109,6 +109,9 @@ bitmap_free(
 	struct bitmap_node	*ext;
 
 	bmap = *bmapp;
+	if (!bmap)
+		return;
+
 	avl_for_each_safe(bmap->bt_tree, node, n) {
 		ext = container_of(node, struct bitmap_node, btn_node);
 		free(ext);
repair/rmap.c +1 -2
index e89bd32d63a953..55c2b0928c52b0 100644
@@ -752,8 +752,7 @@ rmap_commit_agbtree_mappings(
 err:
 	if (agflbp)
 		libxfs_buf_relse(agflbp);
-	if (own_ag_bitmap)
-		bitmap_free(&own_ag_bitmap);
+	bitmap_free(&own_ag_bitmap);
 	return error;
 }
 
scrub/phase5.c +2 -4
index 577dda8064c3a8..52bbbca4b9f06a 100644
@@ -897,10 +897,8 @@ _("Filesystem has errors, skipping connectivity checks."));
 	scrub_report_preen_triggers(ctx);
 out_lock:
 	pthread_mutex_destroy(&ncs.lock);
-	if (ncs.new_deferred)
-		bitmap_free(&ncs.new_deferred);
-	if (ncs.cur_deferred)
-		bitmap_free(&ncs.cur_deferred);
+	bitmap_free(&ncs.new_deferred);
+	bitmap_free(&ncs.cur_deferred);
 	return ret;
 }
 
#32 lore raw

PATCH 2/22 mkfs: rename byte unit conversion macros

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rename these macros so that we can promote the generic ones in the next patch.

---
 mkfs/xfs_mkfs.c |   33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)
mkfs/xfs_mkfs.c +17 -16
index f859626afdda36..e0f0bb28e12159 100644
@@ -18,9 +18,10 @@
 #include "proto.h"
 #include <ini.h>
 
-#define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
-#define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
-#define MEGABYTES(count, blog)	((uint64_t)(count) << (20 - (blog)))
+/* Convert a quantity of mega/giga/terabytes into units of blocks */
+#define TERABLOCKS(count, blog)	((uint64_t)(count) << (40 - (blog)))
+#define GIGABLOCKS(count, blog)	((uint64_t)(count) << (30 - (blog)))
+#define MEGABLOCKS(count, blog)	((uint64_t)(count) << (20 - (blog)))
 
 /*
  * Realistically, the log should never be smaller than 64MB.  Studies by the
@@ -28,7 +29,7 @@
  * latency of the xlog grant head waitqueue when running a heavy metadata
  * update workload when the log size is at least 64MB.
  */
-#define XFS_MIN_REALISTIC_LOG_BLOCKS(blog)	(MEGABYTES(64, (blog)))
+#define XFS_MIN_REALISTIC_LOG_BLOCKS(blog)	(MEGABLOCKS(64, (blog)))
 
 /*
  * Use this macro before we have superblock and mount structure to
@@ -3443,7 +3444,7 @@ validate_supported(
 	 *
 	 * 64MB * (8 / 7) * 4 = 293MB
 	 */
-	if (mp->m_sb.sb_dblocks < MEGABYTES(300, mp->m_sb.sb_blocklog)) {
+	if (mp->m_sb.sb_dblocks < MEGABLOCKS(300, mp->m_sb.sb_blocklog)) {
 		fprintf(stderr,
  _("Filesystem must be larger than 300MB.\n"));
 		usage();
@@ -3559,7 +3560,7 @@ _("%s: Volume reports invalid stripe unit (%d) and stripe width (%d), ignoring.\
 				BBTOB(ft->data.sunit), BBTOB(ft->data.swidth));
 			ft->data.sunit = 0;
 			ft->data.swidth = 0;
-		} else if (cfg->dblocks < GIGABYTES(1, cfg->blocklog)) {
+		} else if (cfg->dblocks < GIGABLOCKS(1, cfg->blocklog)) {
 			/*
 			 * Don't use automatic stripe detection if the device
 			 * size is less than 1GB because the performance gains
@@ -4056,7 +4057,7 @@ calc_concurrency_ag_geometry(
 	 */
 	try_threads = nr_threads;
 	try_agsize = cfg->dblocks / try_threads;
-	if (try_agsize < GIGABYTES(4, cfg->blocklog)) {
+	if (try_agsize < GIGABLOCKS(4, cfg->blocklog)) {
 		do {
 			try_threads--;
 			if (try_threads <= def_agcount) {
@@ -4065,7 +4066,7 @@ calc_concurrency_ag_geometry(
 			}
 
 			try_agsize = cfg->dblocks / try_threads;
-		} while (try_agsize < GIGABYTES(4, cfg->blocklog));
+		} while (try_agsize < GIGABLOCKS(4, cfg->blocklog));
 		goto out;
 	}
 
@@ -4413,7 +4414,7 @@ calc_concurrency_rtgroup_geometry(
 	 */
 	try_threads = nr_threads;
 	try_rgsize = cfg->rtblocks / try_threads;
-	if (try_rgsize < GIGABYTES(4, cfg->blocklog)) {
+	if (try_rgsize < GIGABLOCKS(4, cfg->blocklog)) {
 		do {
 			try_threads--;
 			if (try_threads <= def_rgcount) {
@@ -4422,7 +4423,7 @@ calc_concurrency_rtgroup_geometry(
 			}
 
 			try_rgsize = cfg->rtblocks / try_threads;
-		} while (try_rgsize < GIGABYTES(4, cfg->blocklog));
+		} while (try_rgsize < GIGABLOCKS(4, cfg->blocklog));
 		goto out;
 	}
 
@@ -4516,7 +4517,7 @@ _("rgsize (%s) not a multiple of fs blk size (%d)\n"),
 		 * If nobody specified a realtime device or the rtgroup size,
 		 * try 1TB, rounded down to the nearest rt extent.
 		 */
-		cfg->rgsize = TERABYTES(1, cfg->blocklog);
+		cfg->rgsize = TERABLOCKS(1, cfg->blocklog);
 		cfg->rgsize -= cfg->rgsize % cfg->rtextblocks;
 		cfg->rgcount = 0;
 	} else if (cfg->rtblocks < cfg->rtextblocks * 2) {
@@ -4651,7 +4652,7 @@ _("rgsize (%s) not a multiple of fs blk size (%d)\n"),
 					(cfg->rtblocks % cfg->rgcount != 0);
 		} else {
 			/* 256MB zones just like typical SMR HDDs */
-			cfg->rgsize = MEGABYTES(256, cfg->blocklog);
+			cfg->rgsize = MEGABLOCKS(256, cfg->blocklog);
 			cfg->rgcount = cfg->rtblocks / cfg->rgsize +
 					(cfg->rtblocks % cfg->rgsize != 0);
 		}
@@ -4692,9 +4693,9 @@ calculate_imaxpct(
 	 *  - under  1 TB, use XFS_DFL_IMAXIMUM_PCT (25%).
 	 */
 
-	if (cfg->dblocks < TERABYTES(1, cfg->blocklog))
+	if (cfg->dblocks < TERABLOCKS(1, cfg->blocklog))
 		cfg->imaxpct = XFS_DFL_IMAXIMUM_PCT;
-	else if (cfg->dblocks < TERABYTES(50, cfg->blocklog))
+	else if (cfg->dblocks < TERABLOCKS(50, cfg->blocklog))
 		cfg->imaxpct = 5;
 	else
 		cfg->imaxpct = 1;
@@ -5016,7 +5017,7 @@ calc_concurrency_logblocks(
 	 * If this filesystem is smaller than a gigabyte, there's little to be
 	 * gained from making the log larger.
 	 */
-	if (cfg->dblocks < GIGABYTES(1, cfg->blocklog))
+	if (cfg->dblocks < GIGABLOCKS(1, cfg->blocklog))
 		goto out;
 
 	/*
@@ -5279,7 +5280,7 @@ _("max log size %d smaller than min log size %d, filesystem is too small\n"),
 				XFS_MIN_REALISTIC_LOG_BLOCKS(cfg->blocklog));
 
 		/* And for a tiny filesystem, use the absolute minimum size */
-		if (cfg->dblocks < MEGABYTES(300, cfg->blocklog))
+		if (cfg->dblocks < MEGABLOCKS(300, cfg->blocklog))
 			cfg->logblocks = min_logblocks;
 
 		/* Ensure the chosen size fits within log size requirements */
#34 lore raw

PATCH 3/22 libfrog: lift *BYTES helpers to convert.h

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Move these byte unit conversion macros to convert.h and amend the macros to cast to unsigned long long to avoid shifting issues. Now we can use these same macros throughout the codebase instead of opencoding shifts and possibly suffering from integer shifting problems.

---
 libfrog/convert.h         |    7 +++++++
 db/bmap_inflate.c         |    2 +-
 libfrog/convert.c         |    7 -------
 mdrestore/xfs_mdrestore.c |    5 +++--
 mkfs/xfs_mkfs.c           |    5 +++--
 repair/agbtree.c          |    3 ++-
 repair/bmap_repair.c      |    3 ++-
 repair/xfs_repair.c       |    5 +++--
 scrub/common.c            |   17 +++++++++--------
 scrub/phase8.c            |    3 ++-
 10 files changed, 32 insertions(+), 25 deletions(-)
libfrog/convert.h +7 -0
index 3e5fbe055986a4..223d3e73c9422b 100644
@@ -22,4 +22,11 @@ extern uid_t	uid_from_string(char *user);
 extern gid_t	gid_from_string(char *group);
 extern prid_t	prid_from_string(char *project);
 
+#define EXABYTES(x)	((unsigned long long)(x) << 60)
+#define PETABYTES(x)	((unsigned long long)(x) << 50)
+#define TERABYTES(x)	((unsigned long long)(x) << 40)
+#define GIGABYTES(x)	((unsigned long long)(x) << 30)
+#define MEGABYTES(x)	((unsigned long long)(x) << 20)
+#define KILOBYTES(x)	((unsigned long long)(x) << 10)
+
 #endif	/* __LIBFROG_CONVERT_H__ */
db/bmap_inflate.c +1 -1
index 1de6d3439ab3d3..cc7c197e788d7f 100644
@@ -437,7 +437,7 @@ bmapinflate_f(
 	struct xfs_trans	*tp;
 	char			*p;
 	unsigned long long	nextents = 0;
-	unsigned long long	dirty_bytes = 60U << 20; /* 60MiB */
+	unsigned long long	dirty_bytes = MEGABYTES(60);
 	unsigned long long	dirty_blocks;
 	unsigned int		resblks;
 	bool			estimate = false;
libfrog/convert.c +0 -7
index 0ceeb389682ae1..65b9e47459e234 100644
@@ -173,13 +173,6 @@ cvt_u16(
 	return i;
 }
 
-#define EXABYTES(x)	((long long)(x) << 60)
-#define PETABYTES(x)	((long long)(x) << 50)
-#define TERABYTES(x)	((long long)(x) << 40)
-#define GIGABYTES(x)	((long long)(x) << 30)
-#define MEGABYTES(x)	((long long)(x) << 20)
-#define KILOBYTES(x)	((long long)(x) << 10)
-
 long long
 cvtnum(
 	size_t		blksize,
mdrestore/xfs_mdrestore.c +3 -2
index 8858026c87dc97..ce7f596a6184fe 100644
@@ -8,6 +8,7 @@
 #include "xfs_metadump.h"
 #include <libfrog/platform.h>
 #include "libfrog/div64.h"
+#include "libfrog/convert.h"
 
 union mdrestore_headers {
 	__be32				magic;
@@ -92,10 +93,10 @@ final_print_progress(
 	if (!mdrestore.show_progress)
 		goto done;
 
-	if (bytes_read <= (*cursor << 20))
+	if (bytes_read <= MEGABYTES(*cursor))
 		goto done;
 
-	print_progress("%lld MB read", howmany_64(bytes_read, 1U << 20));
+	print_progress("%lld MB read", howmany_64(bytes_read, MEGABYTES(1)));
 
 done:
 	if (mdrestore.progress_since_warning)
mkfs/xfs_mkfs.c +3 -2
index e0f0bb28e12159..dd8a48c3633ef0 100644
@@ -15,6 +15,7 @@
 #include "libfrog/dahashselftest.h"
 #include "libfrog/fsproperties.h"
 #include "libfrog/zones.h"
+#include "libfrog/convert.h"
 #include "proto.h"
 #include <ini.h>
 
@@ -558,7 +559,7 @@ static struct opt_params iopts = {
 		  .conflicts = { { NULL, LAST_CONFLICT } },
 		  .convert = true,
 		  .minval = 1,
-		  .maxval = 1ULL << 30, /* 1GiB */
+		  .maxval = GIGABYTES(1),
 		  .defaultval = SUBOPT_NEEDS_VAL,
 		},
 	},
@@ -1576,7 +1577,7 @@ discard_blocks(int fd, uint64_t nsectors, int quiet)
 {
 	uint64_t	offset = 0;
 	/* Discard the device 2G at a time */
-	const uint64_t	step = 2ULL << 30;
+	const uint64_t	step = GIGABYTES(2);
 	const uint64_t	count = BBTOB(nsectors);
 
 	/*
repair/agbtree.c +2 -1
index 983b645e1a35a3..fe28e5e94c575e 100644
@@ -6,6 +6,7 @@
 #include <libxfs.h>
 #include "err_protos.h"
 #include "libfrog/bitmap.h"
+#include "libfrog/convert.h"
 #include "slab.h"
 #include "rmap.h"
 #include "incore.h"
@@ -23,7 +24,7 @@ init_rebuild(
 	memset(btr, 0, sizeof(struct bt_rebuild));
 
 	bulkload_init_ag(&btr->newbt, sc, oinfo, NULLFSBLOCK);
-	btr->bload.max_dirty = XFS_B_TO_FSBT(sc->mp, 256U << 10); /* 256K */
+	btr->bload.max_dirty = XFS_B_TO_FSBT(sc->mp, KILOBYTES(256));
 	bulkload_estimate_ag_slack(sc, &btr->bload, est_agfreeblocks);
 }
 
repair/bmap_repair.c +2 -1
index 5d1f639be81ff4..192f189de05217 100644
@@ -15,6 +15,7 @@
 #include "bulkload.h"
 #include "bmap_repair.h"
 #include "libfrog/util.h"
+#include "libfrog/convert.h"
 
 /*
  * Inode Fork Block Mapping (BMBT) Repair
@@ -499,7 +500,7 @@ xrep_bmap_btree_load(
 	rb->bmap_bload.get_records = xrep_bmap_get_records;
 	rb->bmap_bload.claim_block = xrep_bmap_claim_block;
 	rb->bmap_bload.iroot_size = xrep_bmap_iroot_size;
-	rb->bmap_bload.max_dirty = XFS_B_TO_FSBT(sc->mp, 256U << 10); /* 256K */
+	rb->bmap_bload.max_dirty = XFS_B_TO_FSBT(sc->mp, KILOBYTES(256));
 
 	/*
 	 * Always make the btree as small as possible, since we might need the
repair/xfs_repair.c +3 -2
index 7bf75c09b94542..6b97a806384096 100644
@@ -28,6 +28,7 @@
 #include "quotacheck.h"
 #include "rcbag_btree.h"
 #include "rt.h"
+#include "libfrog/convert.h"
 
 /*
  * option tables for getsubopt calls
@@ -1321,8 +1322,8 @@ main(int argc, char **argv)
 		}
 
 		max_mem -= mem_used;
-		if (max_mem >= (1 << 30))
-			max_mem = 1 << 30;
+		if (max_mem >= GIGABYTES(1))
+			max_mem = GIGABYTES(1);
 		libxfs_bhash_size = max_mem / (HASH_CACHE_RATIO *
 				(igeo->inode_cluster_size >> 10));
 		if (libxfs_bhash_size < 512)
scrub/common.c +9 -8
index a567d2a3c2f14e..34d91525928305 100644
@@ -11,6 +11,7 @@
 #include "libfrog/paths.h"
 #include "libfrog/getparents.h"
 #include "libfrog/handle_priv.h"
+#include "libfrog/convert.h"
 #include "xfs_scrub.h"
 #include "common.h"
 #include "progress.h"
@@ -218,18 +219,18 @@ auto_space_units(
 {
 	if (debug > 1)
 		goto no_prefix;
-	if (bytes > (1ULL << 40)) {
+	if (bytes > TERABYTES(1)) {
 		*units = "TiB";
-		return (double)bytes / (1ULL << 40);
-	} else if (bytes > (1ULL << 30)) {
+		return (double)bytes / TERABYTES(1);
+	} else if (bytes > GIGABYTES(1)) {
 		*units = "GiB";
-		return (double)bytes / (1ULL << 30);
-	} else if (bytes > (1ULL << 20)) {
+		return (double)bytes / GIGABYTES(1);
+	} else if (bytes > MEGABYTES(1)) {
 		*units = "MiB";
-		return (double)bytes / (1ULL << 20);
-	} else if (bytes > (1ULL << 10)) {
+		return (double)bytes / MEGABYTES(1);
+	} else if (bytes > KILOBYTES(1)) {
 		*units = "KiB";
-		return (double)bytes / (1ULL << 10);
+		return (double)bytes / KILOBYTES(1);
 	}
 
 no_prefix:
scrub/phase8.c +2 -1
index e8c72d8eb851af..0967832da890e3 100644
@@ -12,6 +12,7 @@
 #include "libfrog/paths.h"
 #include "libfrog/workqueue.h"
 #include "libfrog/histogram.h"
+#include "libfrog/convert.h"
 #include "xfs_scrub.h"
 #include "common.h"
 #include "progress.h"
@@ -51,7 +52,7 @@ fstrim_ok(
  * call so that we can implement decent progress reporting and CPU resource
  * control.  Pick a prime number of gigabytes for interest.
  */
-#define FSTRIM_MAX_BYTES	(11ULL << 30)
+#define FSTRIM_MAX_BYTES	GIGABYTES(11)
 
 /* Trim a certain range of the filesystem. */
 static int
#36 lore raw

PATCH 4/22 xfs_scrub: report truncated devices as media errors

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If we encounter a zero-length read of an xfs device but don't hit any actual media errors, we won't report the truncated device. Fix that. Also fix the mistake that flushing the verify-pools of the log/rt devices doesn't actually cause scrub to abort.

Cc <[email protected]> # v6.13.0
Fixes a6e089903f2f58 ("xfs_scrub: tread zero-length read verify as an IO error")
---
 scrub/phase6.c |    8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
scrub/phase6.c +6 -2
index abf6f9713f1a4d..590e5d23e6b267 100644
@@ -782,9 +782,13 @@ phase6_func(
 	 * If the verify flush didn't work or we found no bad blocks, we're
 	 * done!  No errors detected.
 	 */
-	if (ret || ret2 || ret3)
+	if (ret || ret2 || ret3) {
+		ret |= ret2 | ret3; /* caller only cares about non-zero/zero */
 		goto out_rbad;
-	if (bitmap_empty(vs.d_bad) && bitmap_empty(vs.r_bad))
+	}
+	if (bitmap_empty(vs.d_bad) && !vs.d_trunc &&
+	    bitmap_empty(vs.r_bad) && !vs.r_trunc &&
+	    !vs.l_trunc)
 		goto out_rbad;
 
 	/* Scan the whole dir tree to see what matches the bad extents. */
#38 lore raw

PATCH 5/22 xfs_scrub: fix i18n of the decode_special_owner return value

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

The function decode_special_owner turns a special fsmap owner into a printable string value. However, it does not query the gettext catalog for a local language translation, which leads to annoying multilanguage failure messages. Fix that by adding the appropriate wrappers.

Cc <[email protected]> # v4.15.0
Fixes b364a9c008fc04 ("xfs_scrub: scrub file data blocks")
---
 scrub/phase6.c |   22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
scrub/phase6.c +11 -11
index 590e5d23e6b267..41b41aab7e2578 100644
@@ -123,16 +123,16 @@ struct owner_decode {
 };
 
 static const struct owner_decode special_owners[] = {
-	{XFS_FMR_OWN_FREE,	"free space"},
-	{XFS_FMR_OWN_UNKNOWN,	"unknown owner"},
-	{XFS_FMR_OWN_FS,	"static FS metadata"},
-	{XFS_FMR_OWN_LOG,	"journalling log"},
-	{XFS_FMR_OWN_AG,	"per-AG metadata"},
-	{XFS_FMR_OWN_INOBT,	"inode btree blocks"},
-	{XFS_FMR_OWN_INODES,	"inodes"},
-	{XFS_FMR_OWN_REFC,	"refcount btree"},
-	{XFS_FMR_OWN_COW,	"CoW staging"},
-	{XFS_FMR_OWN_DEFECTIVE,	"bad blocks"},
+	{XFS_FMR_OWN_FREE,	N_("free space")},
+	{XFS_FMR_OWN_UNKNOWN,	N_("unknown owner")},
+	{XFS_FMR_OWN_FS,	N_("static FS metadata")},
+	{XFS_FMR_OWN_LOG,	N_("journalling log")},
+	{XFS_FMR_OWN_AG,	N_("per-AG metadata")},
+	{XFS_FMR_OWN_INOBT,	N_("inode btree blocks")},
+	{XFS_FMR_OWN_INODES,	N_("inodes")},
+	{XFS_FMR_OWN_REFC,	N_("refcount btree")},
+	{XFS_FMR_OWN_COW,	N_("CoW staging")},
+	{XFS_FMR_OWN_DEFECTIVE,	N_("bad blocks")},
 	{0, NULL},
 };
 
@@ -145,7 +145,7 @@ decode_special_owner(
 
 	while (od->descr) {
 		if (od->owner == owner)
-			return od->descr;
+			return _(od->descr);
 		od++;
 	}
 
#40 lore raw

PATCH 6/22 scrub: remove the unused io_disk field in struct read_verify

In reply to Darrick J. Wong

From: Christoph Hellwig <[email protected]>

---
 scrub/read_verify.c |    1 -
 1 file changed, 1 deletion(-)
scrub/read_verify.c +0 -1
index 1219efe2590182..56df7a40d6c07a 100644
@@ -51,7 +51,6 @@ rvp_io_max_size(void)
 
 struct read_verify {
 	void			*io_end_arg;
-	struct disk		*io_disk;
 	uint64_t		io_start;	/* bytes */
 	uint64_t		io_length;	/* bytes */
 };
#41 lore raw

PATCH 7/22 xfs_scrub: move read verification scheduling to phase6.c

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Right now there's a weird coupling between read_verify.c and spacemap.c:
Anyone using a read_verify_pool is required to tell the pool how many threads it's going to use to call read_verify_schedule_io. This is because the read_verify_pool accumulates verification requests on a per-thread basis to try to combine adjacent written regions for media verification. However, the verification requests are made from the phase6.c callback (check_rmap) that is called from the workers created by scrub_scan_all_spacemaps.

Yeah, that's confusing: implementation details of spacemap.c must be inferred by phase6.c and passed to read_verify.c.

Let's fix this by moving the per-thread schedule accumulation to phase6.c before the next patches constrain the number of IO threads sending verification requests to the kernel.

---
 scrub/read_verify.h |   16 +++++--
 scrub/spacemap.h    |    5 ++
 scrub/phase6.c      |   63 ++++++++++++++++++++++-----
 scrub/read_verify.c |  121 ++++++++++++++++++---------------------------------
 4 files changed, 112 insertions(+), 93 deletions(-)
scrub/read_verify.h +12 -4
index 9d34d839c978b4..383823c3b5c640 100644
@@ -10,6 +10,13 @@ struct scrub_ctx;
 struct read_verify_pool;
 struct disk;
 
+struct read_verify_schedule {
+	struct read_verify_pool	*rvp;
+	void			*io_end_arg;
+	uint64_t		io_start;	/* bytes */
+	uint64_t		io_length;	/* bytes */
+};
+
 /* Function called when an IO error happens. */
 typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx,
 		struct disk *disk, uint64_t start, uint64_t length,
@@ -17,15 +24,16 @@ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx,
 
 int read_verify_pool_alloc(struct scrub_ctx *ctx, struct disk *disk,
 		size_t miniosz, read_verify_ioerr_fn_t ioerr_fn,
-		unsigned int submitter_threads,
 		struct read_verify_pool **prvp);
 void read_verify_pool_abort(struct read_verify_pool *rvp);
 int read_verify_pool_flush(struct read_verify_pool *rvp);
 void read_verify_pool_destroy(struct read_verify_pool *rvp);
 
-int read_verify_schedule_io(struct read_verify_pool *rvp, uint64_t start,
-		uint64_t length, void *end_arg);
-int read_verify_force_io(struct read_verify_pool *rvp);
+int read_verify_schedule_now(struct read_verify_schedule *rs);
+bool try_read_verify_schedule_io(struct read_verify_schedule *rs,
+		struct read_verify_pool *rvp, uint64_t start, uint64_t length,
+		void *end_arg);
+
 int read_verify_bytes(struct read_verify_pool *rvp, uint64_t *bytes);
 
 #endif /* XFS_SCRUB_READ_VERIFY_H_ */
scrub/spacemap.h +5 -0
index 51975341b16d6b..759d0f89089a23 100644
@@ -18,4 +18,9 @@ int scrub_iterate_fsmap(struct scrub_ctx *ctx, struct fsmap *keys,
 int scrub_scan_all_spacemaps(struct scrub_ctx *ctx, scrub_fsmap_iter_fn fn,
 		void *arg);
 
+static inline unsigned int scrub_scan_spacemaps_nproc(struct scrub_ctx *ctx)
+{
+	return scrub_nproc(ctx);
+}
+
 #endif /* XFS_SCRUB_SPACEMAP_H_ */
scrub/phase6.c +53 -10

135 lines not on this page · raw message

scrub/read_verify.c +42 -79

213 lines not on this page · raw message

#43 lore raw

PATCH 8/22 scrub: simplify the read_verify_pool_alloc interface

In reply to Darrick J. Wong

From: Christoph Hellwig <[email protected]>

Don't pass the miniosize as that's always the fs block size.

[djwong: rebase atop another read verify cleanup]
---
 scrub/read_verify.h |    2 +-
 scrub/phase6.c      |    9 +++------
 scrub/read_verify.c |   12 ++----------
 3 files changed, 6 insertions(+), 17 deletions(-)
scrub/read_verify.h +1 -1
index 383823c3b5c640..a46244334268de 100644
@@ -23,7 +23,7 @@ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx,
 		int error, void *arg);
 
 int read_verify_pool_alloc(struct scrub_ctx *ctx, struct disk *disk,
-		size_t miniosz, read_verify_ioerr_fn_t ioerr_fn,
+		read_verify_ioerr_fn_t ioerr_fn,
 		struct read_verify_pool **prvp);
 void read_verify_pool_abort(struct read_verify_pool *rvp);
 int read_verify_pool_flush(struct read_verify_pool *rvp);
scrub/phase6.c +3 -6
index a28ebee8e7b272..bfd5659590d603 100644
@@ -761,16 +761,14 @@ phase6_func(
 		goto out_dbad;
 	}
 
-	ret = read_verify_pool_alloc(ctx, ctx->datadev,
-			ctx->mnt.fsgeom.blocksize, remember_ioerr,
+	ret = read_verify_pool_alloc(ctx, ctx->datadev, remember_ioerr,
 			&vs.rvp_data);
 	if (ret) {
 		str_liberror(ctx, ret, _("creating datadev media verifier"));
 		goto out_rbad;
 	}
 	if (ctx->logdev) {
-		ret = read_verify_pool_alloc(ctx, ctx->logdev,
-				ctx->mnt.fsgeom.blocksize, remember_ioerr,
+		ret = read_verify_pool_alloc(ctx, ctx->logdev, remember_ioerr,
 				&vs.rvp_log);
 		if (ret) {
 			str_liberror(ctx, ret,
@@ -779,8 +777,7 @@ phase6_func(
 		}
 	}
 	if (ctx->rtdev) {
-		ret = read_verify_pool_alloc(ctx, ctx->rtdev,
-				ctx->mnt.fsgeom.blocksize, remember_ioerr,
+		ret = read_verify_pool_alloc(ctx, ctx->rtdev, remember_ioerr,
 				&vs.rvp_realtime);
 		if (ret) {
 			str_liberror(ctx, ret,
scrub/read_verify.c +2 -10
index 2447ed272734be..128df2be86898b 100644
@@ -74,14 +74,12 @@ struct read_verify_pool {
  * Create a thread pool to run read verifiers.
  *
  * @disk is the disk we want to verify.
- * @miniosz is the minimum size of an IO to expect (in bytes).
  * @ioerr_fn will be called when IO errors occur.
  */
 int
 read_verify_pool_alloc(
 	struct scrub_ctx		*ctx,
 	struct disk			*disk,
-	size_t				miniosz,
 	read_verify_ioerr_fn_t		ioerr_fn,
 	struct read_verify_pool		**prvp)
 {
@@ -89,13 +87,7 @@ read_verify_pool_alloc(
 	unsigned int			verifier_threads = disk_heads(disk);
 	int				ret;
 
-	/*
-	 * The minimum IO size must be a multiple of the disk sector size
-	 * and a factor of the max io size.
-	 */
-	if (miniosz % disk->d_lbasize)
-		return EINVAL;
-	if (rvp_io_max_size() % miniosz)
+	if (rvp_io_max_size() % ctx->mnt.fsgeom.blocksize)
 		return EINVAL;
 
 	rvp = calloc(1, sizeof(struct read_verify_pool));
@@ -109,7 +101,7 @@ read_verify_pool_alloc(
 	ret = ptcounter_alloc(verifier_threads, &rvp->verified_bytes);
 	if (ret)
 		goto out_buf;
-	rvp->miniosz = miniosz;
+	rvp->miniosz = ctx->mnt.fsgeom.blocksize;
 	rvp->ctx = ctx;
 	rvp->disk = disk;
 	rvp->ioerr_fn = ioerr_fn;
#44 lore raw

PATCH 9/22 xfs_scrub: don't pass the io_end_arg around everywhere

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

The value is the same across all callers and read-verify pools, so let's just pass it into read_verify_pool_alloc instead of making temporary aliases everywhere and increasing memory usage.

---
 scrub/read_verify.h |    6 ++----
 scrub/phase6.c      |   10 +++++-----
 scrub/read_verify.c |   15 +++++++--------
 3 files changed, 14 insertions(+), 17 deletions(-)
scrub/read_verify.h +2 -4
index a46244334268de..165ad8f7af7b6c 100644
@@ -12,7 +12,6 @@ struct disk;
 
 struct read_verify_schedule {
 	struct read_verify_pool	*rvp;
-	void			*io_end_arg;
 	uint64_t		io_start;	/* bytes */
 	uint64_t		io_length;	/* bytes */
 };
@@ -23,7 +22,7 @@ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx,
 		int error, void *arg);
 
 int read_verify_pool_alloc(struct scrub_ctx *ctx, struct disk *disk,
-		read_verify_ioerr_fn_t ioerr_fn,
+		read_verify_ioerr_fn_t ioerr_fn, void *ioerr_arg,
 		struct read_verify_pool **prvp);
 void read_verify_pool_abort(struct read_verify_pool *rvp);
 int read_verify_pool_flush(struct read_verify_pool *rvp);
@@ -31,8 +30,7 @@ void read_verify_pool_destroy(struct read_verify_pool *rvp);
 
 int read_verify_schedule_now(struct read_verify_schedule *rs);
 bool try_read_verify_schedule_io(struct read_verify_schedule *rs,
-		struct read_verify_pool *rvp, uint64_t start, uint64_t length,
-		void *end_arg);
+		struct read_verify_pool *rvp, uint64_t start, uint64_t length);
 
 int read_verify_bytes(struct read_verify_pool *rvp, uint64_t *bytes);
 
scrub/phase6.c +5 -5
index bfd5659590d603..3e6a236d010a6b 100644
@@ -645,7 +645,7 @@ check_rmap(
 
 	/* Schedule the read verify command for (eventual) running. */
 	scheduled = try_read_verify_schedule_io(rs, rvp, map->fmr_physical,
-			map->fmr_length, vs);
+			map->fmr_length);
 	if (scheduled)
 		return 0;
 
@@ -656,7 +656,7 @@ check_rmap(
 	}
 
 	scheduled = try_read_verify_schedule_io(rs, rvp, map->fmr_physical,
-			map->fmr_length, vs);
+			map->fmr_length);
 	assert(scheduled);
 	return 0;
 }
@@ -761,7 +761,7 @@ phase6_func(
 		goto out_dbad;
 	}
 
-	ret = read_verify_pool_alloc(ctx, ctx->datadev, remember_ioerr,
+	ret = read_verify_pool_alloc(ctx, ctx->datadev, remember_ioerr, &vs,
 			&vs.rvp_data);
 	if (ret) {
 		str_liberror(ctx, ret, _("creating datadev media verifier"));
@@ -769,7 +769,7 @@ phase6_func(
 	}
 	if (ctx->logdev) {
 		ret = read_verify_pool_alloc(ctx, ctx->logdev, remember_ioerr,
-				&vs.rvp_log);
+				&vs, &vs.rvp_log);
 		if (ret) {
 			str_liberror(ctx, ret,
 					_("creating logdev media verifier"));
@@ -778,7 +778,7 @@ phase6_func(
 	}
 	if (ctx->rtdev) {
 		ret = read_verify_pool_alloc(ctx, ctx->rtdev, remember_ioerr,
-				&vs.rvp_realtime);
+				&vs, &vs.rvp_realtime);
 		if (ret) {
 			str_liberror(ctx, ret,
 					_("creating rtdev media verifier"));
scrub/read_verify.c +7 -8
index 128df2be86898b..bcd923e5f3cbc3 100644
@@ -49,7 +49,6 @@ rvp_io_max_size(void)
 #define RVP_IO_BATCH_LOCALITY	(65536)
 
 struct read_verify {
-	void			*io_end_arg;
 	uint64_t		io_start;	/* bytes */
 	uint64_t		io_length;	/* bytes */
 };
@@ -60,6 +59,7 @@ struct read_verify_pool {
 	void			*readbuf;	/* read buffer */
 	struct ptcounter	*verified_bytes;
 	struct disk		*disk;		/* which disk? */
+	void			*ioerr_arg;
 	read_verify_ioerr_fn_t	ioerr_fn;	/* io error callback */
 	size_t			miniosz;	/* minimum io size, bytes */
 
@@ -81,6 +81,7 @@ read_verify_pool_alloc(
 	struct scrub_ctx		*ctx,
 	struct disk			*disk,
 	read_verify_ioerr_fn_t		ioerr_fn,
+	void				*ioerr_arg,
 	struct read_verify_pool		**prvp)
 {
 	struct read_verify_pool		*rvp;
@@ -105,6 +106,7 @@ read_verify_pool_alloc(
 	rvp->ctx = ctx;
 	rvp->disk = disk;
 	rvp->ioerr_fn = ioerr_fn;
+	rvp->ioerr_arg = ioerr_arg;
 	ret = -workqueue_create(&rvp->wq, (struct xfs_mount *)rvp,
 			verifier_threads == 1 ? 0 : verifier_threads);
 	if (ret)
@@ -223,14 +225,14 @@ read_verify(
 					rvp->disk->d_fd, rv->io_start, sz,
 					read_error);
 			rvp->ioerr_fn(rvp->ctx, rvp->disk, rv->io_start, sz,
-					read_error, rv->io_end_arg);
+					read_error, rvp->ioerr_arg);
 		} else if (sz == 0) {
 			/* No bytes at all?  Did we hit the end of the disk? */
 			dbg_printf("EOF %d @ %"PRIu64" %zu err %d\n",
 					rvp->disk->d_fd, rv->io_start, sz,
 					read_error);
 			rvp->ioerr_fn(rvp->ctx, rvp->disk, rv->io_start, sz,
-					read_error, rv->io_end_arg);
+					read_error, rvp->ioerr_arg);
 			break;
 		} else if (sz < len) {
 			/*
@@ -291,7 +293,6 @@ read_verify_schedule_now(
 		return errno;
 	}
 
-	tmp->io_end_arg = rs->io_end_arg;
 	tmp->io_start = rs->io_start;
 	tmp->io_length = rs->io_length;
 
@@ -318,8 +319,7 @@ try_read_verify_schedule_io(
 	struct read_verify_schedule	*rs,
 	struct read_verify_pool		*rvp,
 	uint64_t			start,
-	uint64_t			length,
-	void				*end_arg)
+	uint64_t			length)
 {
 	uint64_t			req_end;
 	uint64_t			rv_end;
@@ -338,7 +338,6 @@ try_read_verify_schedule_io(
 		rs->rvp = rvp;
 		rs->io_start = start;
 		rs->io_length = length;
-		rs->io_end_arg = end_arg;
 
 		return true;
 	}
@@ -348,7 +347,7 @@ try_read_verify_schedule_io(
 	 * reporting is the same, and the two extents are close,
 	 * we can combine them.
 	 */
-	if (rs->rvp == rvp && rs->io_length > 0 && end_arg == rs->io_end_arg &&
+	if (rs->rvp == rvp && rs->io_length > 0 &&
 	    ((start >= rs->io_start && start <= rv_end + RVP_IO_BATCH_LOCALITY) ||
 	     (rs->io_start >= start &&
 	      rs->io_start <= req_end + RVP_IO_BATCH_LOCALITY))) {
#46 lore raw

PATCH 10/22 scrub: use enum xfs_device for read verification

In reply to Darrick J. Wong

From: Christoph Hellwig <[email protected]>

Passing the disk pointer and then translating back to an index is a bit confusing. Rewrite the read_verify and related code to pass around an enum xfs_device and use that to identify the device.

The disks are placed in an array so that they can be trivially indexed using the xfs device. Because the value start at 1 this adds an unused slot, but such a minor waste does not matter in the overall scheme of things.

---
 scrub/read_verify.h |    5 +--
 scrub/xfs_scrub.h   |    6 +--
 scrub/phase1.c      |   26 +++++++------
 scrub/phase6.c      |  101 ++++++++++++++++++++++++++-------------------------
 scrub/read_verify.c |   38 +++++++++----------
 5 files changed, 86 insertions(+), 90 deletions(-)
scrub/read_verify.h +2 -3
index 165ad8f7af7b6c..e4b36f6aaa20e1 100644
@@ -8,7 +8,6 @@
 
 struct scrub_ctx;
 struct read_verify_pool;
-struct disk;
 
 struct read_verify_schedule {
 	struct read_verify_pool	*rvp;
@@ -18,10 +17,10 @@ struct read_verify_schedule {
 
 /* Function called when an IO error happens. */
 typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx,
-		struct disk *disk, uint64_t start, uint64_t length,
+		enum xfs_device dev, uint64_t start, uint64_t length,
 		int error, void *arg);
 
-int read_verify_pool_alloc(struct scrub_ctx *ctx, struct disk *disk,
+int read_verify_pool_alloc(struct scrub_ctx *ctx, enum xfs_device dev,
 		read_verify_ioerr_fn_t ioerr_fn, void *ioerr_arg,
 		struct read_verify_pool **prvp);
 void read_verify_pool_abort(struct read_verify_pool *rvp);
scrub/xfs_scrub.h +2 -4
index 041c0fadaa93c0..a7e67c37469beb 100644
@@ -57,10 +57,8 @@ struct scrub_ctx {
 	struct statvfs		mnt_sv;
 	struct statfs		mnt_sf;
 
-	/* Open block devices */
-	struct disk		*datadev;
-	struct disk		*logdev;
-	struct disk		*rtdev;
+	/* Open block devices for legacy verify */
+	struct disk		*verify_disks[XFS_DEV_RT + 1];
 
 	/* What does the user want us to do? */
 	enum scrub_mode		mode;
scrub/phase1.c +13 -13
index 10e9aa1892b701..954a62b7e8d711 100644
@@ -96,12 +96,12 @@ scrub_cleanup(
 
 	if (ctx->fshandle)
 		free_handle(ctx->fshandle, ctx->fshandle_len);
-	if (ctx->rtdev)
-		disk_close(ctx->rtdev);
-	if (ctx->logdev)
-		disk_close(ctx->logdev);
-	if (ctx->datadev)
-		disk_close(ctx->datadev);
+	if (ctx->verify_disks[XFS_DEV_DATA])
+		disk_close(ctx->verify_disks[XFS_DEV_DATA]);
+	if (ctx->verify_disks[XFS_DEV_LOG])
+		disk_close(ctx->verify_disks[XFS_DEV_LOG]);
+	if (ctx->verify_disks[XFS_DEV_RT])
+		disk_close(ctx->verify_disks[XFS_DEV_RT]);
 	fshandle_destroy();
 	error = -xfd_close(&ctx->mnt);
 	if (error)
@@ -349,13 +349,13 @@ _("Unable to find realtime device path."));
 	}
 
 	/* Open the raw devices. */
-	ctx->datadev = disk_open(ctx->fsinfo.fs_name);
-	if (!ctx->datadev) {
+	ctx->verify_disks[XFS_DEV_DATA] = disk_open(ctx->fsinfo.fs_name);
+	if (!ctx->verify_disks[XFS_DEV_DATA]) {
 		str_error(ctx, ctx->mntpoint, _("Unable to open data device."));
 		return ECANCELED;
 	}
 
-	ctx->nr_io_threads = disk_heads(ctx->datadev);
+	ctx->nr_io_threads = disk_heads(ctx->verify_disks[XFS_DEV_DATA]);
 	if (verbose) {
 		fprintf(stdout, _("%s: using %d threads to scrub.\n"),
 				ctx->mntpoint, scrub_nproc(ctx));
@@ -363,16 +363,16 @@ _("Unable to find realtime device path."));
 	}
 
 	if (ctx->fsinfo.fs_log) {
-		ctx->logdev = disk_open(ctx->fsinfo.fs_log);
-		if (!ctx->logdev) {
+		ctx->verify_disks[XFS_DEV_LOG] = disk_open(ctx->fsinfo.fs_log);
+		if (!ctx->verify_disks[XFS_DEV_LOG]) {
 			str_error(ctx, ctx->mntpoint,
 				_("Unable to open external log device."));
 			return ECANCELED;
 		}
 	}
 	if (ctx->fsinfo.fs_rt) {
-		ctx->rtdev = disk_open(ctx->fsinfo.fs_rt);
-		if (!ctx->rtdev) {
+		ctx->verify_disks[XFS_DEV_RT] = disk_open(ctx->fsinfo.fs_rt);
+		if (!ctx->verify_disks[XFS_DEV_RT]) {
 			str_error(ctx, ctx->mntpoint,
 				_("Unable to open realtime device."));
 			return ECANCELED;
scrub/phase6.c +51 -50

192 lines not on this page · raw message

scrub/read_verify.c +18 -20
index bcd923e5f3cbc3..9b692a923dfdf3 100644
@@ -58,10 +58,10 @@ struct read_verify_pool {
 	struct scrub_ctx	*ctx;		/* scrub context */
 	void			*readbuf;	/* read buffer */
 	struct ptcounter	*verified_bytes;
-	struct disk		*disk;		/* which disk? */
 	void			*ioerr_arg;
 	read_verify_ioerr_fn_t	ioerr_fn;	/* io error callback */
 	size_t			miniosz;	/* minimum io size, bytes */
+	enum xfs_device		dev;		/* which device? */
 
 	/*
 	 * Store a runtime error code here so that we can stop the pool and
@@ -73,19 +73,19 @@ struct read_verify_pool {
 /*
  * Create a thread pool to run read verifiers.
  *
- * @disk is the disk we want to verify.
  * @ioerr_fn will be called when IO errors occur.
  */
 int
 read_verify_pool_alloc(
 	struct scrub_ctx		*ctx,
-	struct disk			*disk,
+	enum xfs_device			dev,
 	read_verify_ioerr_fn_t		ioerr_fn,
 	void				*ioerr_arg,
 	struct read_verify_pool		**prvp)
 {
 	struct read_verify_pool		*rvp;
-	unsigned int			verifier_threads = disk_heads(disk);
+	unsigned int			verifier_threads =
+		disk_heads(ctx->verify_disks[XFS_DEV_DATA]);
 	int				ret;
 
 	if (rvp_io_max_size() % ctx->mnt.fsgeom.blocksize)
@@ -104,7 +104,7 @@ read_verify_pool_alloc(
 		goto out_buf;
 	rvp->miniosz = ctx->mnt.fsgeom.blocksize;
 	rvp->ctx = ctx;
-	rvp->disk = disk;
+	rvp->dev = dev;
 	rvp->ioerr_fn = ioerr_fn;
 	rvp->ioerr_arg = ioerr_arg;
 	ret = -workqueue_create(&rvp->wq, (struct xfs_mount *)rvp,
@@ -179,10 +179,10 @@ read_verify(
 	while (rv->io_length > 0) {
 		read_error = 0;
 		len = min(rv->io_length, io_max_size);
-		dbg_printf("diskverify %d %"PRIu64" %zu\n", rvp->disk->d_fd,
+		dbg_printf("diskverify %u %"PRIu64" %zu\n", rvp->dev,
 				rv->io_start, len);
-		sz = disk_read_verify(rvp->disk, rvp->readbuf, rv->io_start,
-				len);
+		sz = disk_read_verify(rvp->ctx->verify_disks[rvp->dev],
+				rvp->readbuf, rv->io_start, len);
 		if (sz == len && io_max_size < rvp->miniosz) {
 			/*
 			 * If the verify request was 100% successful and less
@@ -221,17 +221,15 @@ read_verify(
 			 * io_start to the next miniosz block.
 			 */
 			sz = rvp->miniosz - (rv->io_start % rvp->miniosz);
-			dbg_printf("IOERR %d @ %"PRIu64" %zu err %d\n",
-					rvp->disk->d_fd, rv->io_start, sz,
-					read_error);
-			rvp->ioerr_fn(rvp->ctx, rvp->disk, rv->io_start, sz,
+			dbg_printf("IOERR %u @ %"PRIu64" %zu err %d\n",
+					rvp->dev, rv->io_start, sz, read_error);
+			rvp->ioerr_fn(rvp->ctx, rvp->dev, rv->io_start, sz,
 					read_error, rvp->ioerr_arg);
 		} else if (sz == 0) {
 			/* No bytes at all?  Did we hit the end of the disk? */
-			dbg_printf("EOF %d @ %"PRIu64" %zu err %d\n",
-					rvp->disk->d_fd, rv->io_start, sz,
-					read_error);
-			rvp->ioerr_fn(rvp->ctx, rvp->disk, rv->io_start, sz,
+			dbg_printf("EOF %u @ %"PRIu64" %zu err %d\n",
+					rvp->dev, rv->io_start, sz, read_error);
+			rvp->ioerr_fn(rvp->ctx, rvp->dev, rv->io_start, sz,
 					read_error, rvp->ioerr_arg);
 			break;
 		} else if (sz < len) {
@@ -245,8 +243,8 @@ read_verify(
 			 * next full block.
 			 */
 			io_max_size = rvp->miniosz - (sz % rvp->miniosz);
-			dbg_printf("SHORT %d READ @ %"PRIu64" %zu try for %zd\n",
-					rvp->disk->d_fd, rv->io_start, sz,
+			dbg_printf("SHORT %u READ @ %"PRIu64" %zu try for %zd\n",
+					rvp->dev, rv->io_start, sz,
 					io_max_size);
 		} else {
 			/* We should never get back more bytes than we asked. */
@@ -279,8 +277,8 @@ read_verify_schedule_now(
 	if (!rvp)
 		return 0;
 
-	dbg_printf("verify fd %d start %"PRIu64" len %"PRIu64"\n",
-			rvp->disk->d_fd, rs->io_start, rs->io_length);
+	dbg_printf("verify dev %u start %"PRIu64" len %"PRIu64"\n",
+			rvp->dev, rs->io_start, rs->io_length);
 
 	/* Worker thread saw a runtime error, don't queue more. */
 	if (rvp->runtime_error)
#47 lore raw

PATCH 11/22 xfs_scrub: rename nr_io_threads

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

This variable really describes the number of threads that we should start up to scan metadata. Rename it to reduce confusion with the media verification code, which also initiates IO.

---
 scrub/xfs_scrub.h |    2 +-
 scrub/common.c    |    2 +-
 scrub/phase1.c    |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
scrub/xfs_scrub.h +1 -1
index a7e67c37469beb..3009a7cd2d97d0 100644
@@ -70,7 +70,7 @@ struct scrub_ctx {
 	struct xfs_fd		mnt;
 
 	/* Number of threads for metadata scrubbing */
-	unsigned int		nr_io_threads;
+	unsigned int		nr_scan_threads;
 
 	/* XFS specific geometry */
 	struct fs_path		fsinfo;
scrub/common.c +1 -1
index 34d91525928305..95bc0c091eae27 100644
@@ -275,7 +275,7 @@ scrub_nproc(
 {
 	if (force_nr_threads)
 		return force_nr_threads;
-	return ctx->nr_io_threads;
+	return ctx->nr_scan_threads;
 }
 
 /*
scrub/phase1.c +1 -1
index 954a62b7e8d711..16607af4c5f025 100644
@@ -355,7 +355,7 @@ _("Unable to find realtime device path."));
 		return ECANCELED;
 	}
 
-	ctx->nr_io_threads = disk_heads(ctx->verify_disks[XFS_DEV_DATA]);
+	ctx->nr_scan_threads = disk_heads(ctx->verify_disks[XFS_DEV_DATA]);
 	if (verbose) {
 		fprintf(stdout, _("%s: using %d threads to scrub.\n"),
 				ctx->mntpoint, scrub_nproc(ctx));
#49 lore raw

PATCH 12/22 scrub: simplify verifier threads calculation

In reply to Darrick J. Wong

From: Christoph Hellwig <[email protected]>

Throwing all CPUs at verifying seems like a bad idea for foreground scrub, as does abusing I/O opt/min as that says absolutely nothing about parallelism. I can't really think of a better way than manually configuring this except maybe kernel hints. But the current decisions are not good defaults, and also are the only user of struct disk for file systems using the kernel verify ioctl.

The best default seems to be 8, because verification speed increases diminish above that level. Note that background service mode now obeys the thread count restrictions.

[djwong: rebase atop previous patches, leave disk_heads() alone]
---
 scrub/read_verify.h |    2 ++
 scrub/phase6.c      |   10 +++++-----
 scrub/read_verify.c |   20 ++++++++++++++++++--
 3 files changed, 25 insertions(+), 7 deletions(-)
scrub/read_verify.h +2 -0
index e4b36f6aaa20e1..6a338abb896bc3 100644
@@ -33,4 +33,6 @@ bool try_read_verify_schedule_io(struct read_verify_schedule *rs,
 
 int read_verify_bytes(struct read_verify_pool *rvp, uint64_t *bytes);
 
+unsigned int read_verify_nproc(struct scrub_ctx *ctx);
+
 #endif /* XFS_SCRUB_READ_VERIFY_H_ */
scrub/phase6.c +5 -5
index 02b25fe73aa656..bae4d7c5d92ce0 100644
@@ -889,11 +889,11 @@ phase6_estimate(
 	 * can contribute to the progress counter.  Hence we need to set
 	 * nr_threads appropriately to handle that many threads.
 	 */
-	*nr_threads = disk_heads(ctx->verify_disks[XFS_DEV_DATA]);
-	if (ctx->verify_disks[XFS_DEV_RT])
-		*nr_threads += disk_heads(ctx->verify_disks[XFS_DEV_RT]);
-	if (ctx->verify_disks[XFS_DEV_LOG])
-		*nr_threads += disk_heads(ctx->verify_disks[XFS_DEV_LOG]);
+	*nr_threads = read_verify_nproc(ctx);
+	if (ctx->fsinfo.fs_rt)
+		*nr_threads = read_verify_nproc(ctx);
+	if (ctx->fsinfo.fs_log)
+		*nr_threads = read_verify_nproc(ctx);
 	*rshift = 20;
 	return 0;
 }
scrub/read_verify.c +18 -2
index 9b692a923dfdf3..ba04ad3684ebd7 100644
@@ -70,6 +70,22 @@ struct read_verify_pool {
 	int			runtime_error;
 };
 
+unsigned int
+read_verify_nproc(
+	struct scrub_ctx		*ctx)
+{
+	if (force_nr_threads)
+		return force_nr_threads;
+
+	/*
+	 * Throwing all CPUs at verifying seems like a bad idea for foreground
+	 * scrub, as does abusing I/O opt/min as that says absolutely nothing
+	 * about parallelism.  The authors observed diminishing returns on
+	 * verification speed past 8 IO threads, so that's the default.
+	 */
+	return 8;
+}
+
 /*
  * Create a thread pool to run read verifiers.
  *
@@ -84,8 +100,8 @@ read_verify_pool_alloc(
 	struct read_verify_pool		**prvp)
 {
 	struct read_verify_pool		*rvp;
-	unsigned int			verifier_threads =
-		disk_heads(ctx->verify_disks[XFS_DEV_DATA]);
+	const unsigned int		verifier_threads =
+		read_verify_nproc(ctx);
 	int				ret;
 
 	if (rvp_io_max_size() % ctx->mnt.fsgeom.blocksize)
#50 lore raw

PATCH 13/22 xfs_scrub: move disk media verification error injection

In reply to Darrick J. Wong

From: Christoph Hellwig <[email protected]>

This isn't really disk-related since it's a knob to make the read_verify_pool pretend that the media is defective. Move this code before we add a new media verify path that doesn't require the disk abstraction.

[djwong: split off from another hch patch, create a new commit message]
---
 scrub/disk.c        |   71 -------------------------------------------
 scrub/read_verify.c |   85 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 83 insertions(+), 73 deletions(-)
scrub/disk.c +0 -71
index 2cf84d91887587..afce801d3f2076 100644
@@ -266,63 +266,6 @@ disk_close(
 #define LBASIZE(d)		(1ULL << (d)->d_lbalog)
 #define BTOLBA(d, bytes)	(((uint64_t)(bytes) + LBASIZE(d) - 1) >> (d)->d_lbalog)
 
-/* Simulate disk errors. */
-static int
-disk_simulate_read_error(
-	struct disk		*disk,
-	uint64_t		start,
-	uint64_t		*length)
-{
-	static int64_t		interval;
-	uint64_t		start_interval;
-
-	/* Simulated disk errors are disabled. */
-	if (interval < 0)
-		return 0;
-
-	/* Figure out the disk read error interval. */
-	if (interval == 0) {
-		char		*p;
-
-		/* Pretend there's bad media every so often, in bytes. */
-		p = getenv("XFS_SCRUB_DISK_ERROR_INTERVAL");
-		if (p == NULL) {
-			interval = -1;
-			return 0;
-		}
-		interval = strtoull(p, NULL, 10);
-		interval &= ~((1U << disk->d_lbalog) - 1);
-	}
-	if (interval <= 0) {
-		interval = -1;
-		return 0;
-	}
-
-	/*
-	 * We simulate disk errors by pretending that there are media errors at
-	 * predetermined intervals across the disk.  If a read verify request
-	 * crosses one of those intervals we shorten it so that the next read
-	 * will start on an interval threshold.  If the read verify request
-	 * starts on an interval threshold, we send back EIO as if it had
-	 * failed.
-	 */
-	if ((start % interval) == 0) {
-		dbg_printf("fd %d: simulating disk error at %"PRIu64".\n",
-				disk->d_fd, start);
-		return EIO;
-	}
-
-	start_interval = start / interval;
-	if (start_interval != (start + *length) / interval) {
-		*length = ((start_interval + 1) * interval) - start;
-		dbg_printf(
-"fd %d: simulating short read at %"PRIu64" to length %"PRIu64".\n",
-				disk->d_fd, start, *length);
-	}
-
-	return 0;
-}
-
 /* Read-verify an extent of a disk device. */
 ssize_t
 disk_read_verify(
@@ -331,20 +274,6 @@ disk_read_verify(
 	uint64_t		start,
 	uint64_t		length)
 {
-	if (debug) {
-		int		ret;
-
-		ret = disk_simulate_read_error(disk, start, &length);
-		if (ret) {
-			errno = ret;
-			return -1;
-		}
-
-		/* Don't actually issue the IO */
-		if (getenv("XFS_SCRUB_DISK_VERIFY_SKIP"))
-			return length;
-	}
-
 	/* Convert to logical block size. */
 	if (disk->d_flags & DISK_FLAG_SCSI_VERIFY)
 		return disk_scsi_verify(disk, BTOLBAT(disk, start),
scrub/read_verify.c +83 -2

97 lines not on this page · raw message

#51 lore raw

PATCH 14/22 xfs_scrub: use the verify media ioctl during phase 6 if possible

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the kernel suppots the XFS_IOC_VERIFY_MEDIA ioctl, use that to perform the phase 6 media scan instead of pwrite or the SCSI VERIFY command. This enables better integration with xfs_healer and fsnotify; and reduces the amount of work that userspace has to do.

---
 scrub/xfs_scrub.h   |    1 +
 scrub/phase1.c      |   18 ++++++++++++++++++
 scrub/read_verify.c |   44 ++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 59 insertions(+), 4 deletions(-)
scrub/xfs_scrub.h +1 -0
index 3009a7cd2d97d0..851b4f37db48e4 100644
@@ -79,6 +79,7 @@ struct scrub_ctx {
 
 	/* Data block read verification buffer */
 	void			*readbuf;
+	bool			no_verify_ioctl;
 
 	/* Mutable scrub state; use lock. */
 	pthread_mutex_t		lock;
scrub/phase1.c +18 -0
index 16607af4c5f025..abc594898bf2c5 100644
@@ -213,6 +213,22 @@ mode_from_autofsck(
 	goto summarize;
 }
 
+/* Does the XFS driver support media scanning its own disks? */
+static void
+configure_xfs_verify(
+	struct scrub_ctx	*ctx)
+{
+	struct xfs_verify_media	me = {
+		/* just probe for support using an empty range */
+		.me_start_daddr	= 0,
+		.me_end_daddr	= 0,
+		.me_dev		= XFS_DEV_DATA,
+	};
+
+	if (ioctl(ctx->mnt.fd, XFS_IOC_VERIFY_MEDIA, &me))
+		ctx->no_verify_ioctl = true;
+}
+
 /*
  * Bind to the mountpoint, read the XFS geometry, bind to the block devices.
  * Anything we've already built will be cleaned up by scrub_cleanup.
@@ -379,6 +395,8 @@ _("Unable to find realtime device path."));
 		}
 	}
 
+	configure_xfs_verify(ctx);
+
 	/*
 	 * Everything's set up, which means any failures recorded after
 	 * this point are most probably corruption errors (as opposed to
scrub/read_verify.c +40 -4

67 lines not on this page · raw message

#52 lore raw

PATCH 15/22 scrub: don't allocate disk for ioctl-based media verify

In reply to Darrick J. Wong

From: Christoph Hellwig <[email protected]>

When the kernel provides the data verification ioctl there is no point in allocating struct disk and thus opening the underlying devices. Refactor the code so that we don't have to do that, with the added benefit of keeping the read verification self-contained in read_verify.c for the case where the kernel provides the ioctl.

---
 scrub/phase1.c |   65 +++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 45 insertions(+), 20 deletions(-)
scrub/phase1.c +45 -20
index abc594898bf2c5..f4d50d016c2ebd 100644
@@ -213,8 +213,39 @@ mode_from_autofsck(
 	goto summarize;
 }
 
+/*
+ * We can't do XFS_IOC_VERIFY_MEDIA media verification, so we need to fall back
+ * to reading the disk.  We already opened the data device, now we need to open
+ * the rt and log devices for media verification.
+ */
+static int
+configure_xfs_verify_fallback(
+	struct scrub_ctx	*ctx)
+{
+	if (ctx->fsinfo.fs_log) {
+		ctx->verify_disks[XFS_DEV_LOG] = disk_open(ctx->fsinfo.fs_log);
+		if (!ctx->verify_disks[XFS_DEV_LOG]) {
+			str_error(ctx, ctx->mntpoint,
+				_("Unable to open external log device."));
+			return ECANCELED;
+		}
+	}
+
+	if (ctx->fsinfo.fs_rt) {
+		ctx->verify_disks[XFS_DEV_RT] = disk_open(ctx->fsinfo.fs_rt);
+		if (!ctx->verify_disks[XFS_DEV_RT]) {
+			str_error(ctx, ctx->mntpoint,
+				_("Unable to open realtime device."));
+			return ECANCELED;
+		}
+	}
+
+	ctx->no_verify_ioctl = true;
+	return 0;
+}
+
 /* Does the XFS driver support media scanning its own disks? */
-static void
+static bool
 configure_xfs_verify(
 	struct scrub_ctx	*ctx)
 {
@@ -225,8 +256,7 @@ configure_xfs_verify(
 		.me_dev		= XFS_DEV_DATA,
 	};
 
-	if (ioctl(ctx->mnt.fd, XFS_IOC_VERIFY_MEDIA, &me))
-		ctx->no_verify_ioctl = true;
+	return ioctl(ctx->mnt.fd, XFS_IOC_VERIFY_MEDIA, &me) == 0;
 }
 
 /*
@@ -378,24 +408,19 @@ _("Unable to find realtime device path."));
 		fflush(stdout);
 	}
 
-	if (ctx->fsinfo.fs_log) {
-		ctx->verify_disks[XFS_DEV_LOG] = disk_open(ctx->fsinfo.fs_log);
-		if (!ctx->verify_disks[XFS_DEV_LOG]) {
-			str_error(ctx, ctx->mntpoint,
-				_("Unable to open external log device."));
-			return ECANCELED;
-		}
+	if (configure_xfs_verify(ctx)) {
+		/*
+		 * ioctl-based media verification is enabled and we already set
+		 * nr_io_threads from the data device, so we no longer need to
+		 * keep this open.
+		 */
+		disk_close(ctx->verify_disks[XFS_DEV_DATA]);
+		ctx->verify_disks[XFS_DEV_DATA] = NULL;
+	} else {
+		error = configure_xfs_verify_fallback(ctx);
+		if (error)
+			return error;
 	}
-	if (ctx->fsinfo.fs_rt) {
-		ctx->verify_disks[XFS_DEV_RT] = disk_open(ctx->fsinfo.fs_rt);
-		if (!ctx->verify_disks[XFS_DEV_RT]) {
-			str_error(ctx, ctx->mntpoint,
-				_("Unable to open realtime device."));
-			return ECANCELED;
-		}
-	}
-
-	configure_xfs_verify(ctx);
 
 	/*
 	 * Everything's set up, which means any failures recorded after
#53 lore raw

PATCH 16/22 xfs_scrub: perform media scanning of the log region

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Scan the log area for media errors because a defect in a region could prevent the user from being able to perform log recovery.

---
 scrub/phase6.c |   37 ++++++++++++++++++++++++++++++-------
 1 file changed, 30 insertions(+), 7 deletions(-)
scrub/phase6.c +30 -7
index bae4d7c5d92ce0..c1ce222d4ce99f 100644
@@ -47,6 +47,7 @@ struct media_verify_state {
 	struct read_verify_pool	*rvp_realtime;
 	struct bitmap		*d_bad;		/* bytes */
 	struct bitmap		*r_bad;		/* bytes */
+	struct bitmap		*l_bad;		/* bytes */
 	bool			d_trunc:1;
 	bool			r_trunc:1;
 	bool			l_trunc:1;
@@ -109,6 +110,8 @@ bitmap_for_disk(
 		return vs->d_bad;
 	case XFS_DEV_RT:
 		return vs->r_bad;
+	case XFS_DEV_LOG:
+		return vs->l_bad;
 	default:
 		return NULL;
 	}
@@ -571,6 +574,12 @@ report_all_media_errors(
 		return ret;
 	}
 
+	ret = report_disk_ioerrs(ctx, vs, XFS_DEV_LOG);
+	if (ret) {
+		str_liberror(ctx, ret, _("walking log device io errors"));
+		return ret;
+	}
+
 	ret = report_disk_ioerrs(ctx, vs, XFS_DEV_RT);
 	if (ret) {
 		str_liberror(ctx, ret, _("walking rtdev io errors"));
@@ -617,9 +626,14 @@ check_rmap(
 			map->fmr_flags);
 
 	/* "Unknown" extents should be verified; they could be data. */
-	if ((map->fmr_flags & FMR_OF_SPECIAL_OWNER) &&
-			map->fmr_owner == XFS_FMR_OWN_UNKNOWN)
-		map->fmr_flags &= ~FMR_OF_SPECIAL_OWNER;
+	if ((map->fmr_flags & FMR_OF_SPECIAL_OWNER)) {
+		switch (map->fmr_owner) {
+		case XFS_FMR_OWN_UNKNOWN:
+		case XFS_FMR_OWN_LOG:
+			map->fmr_flags &= ~FMR_OF_SPECIAL_OWNER;
+			break;
+		}
+	}
 
 	/*
 	 * We only care about read-verifying data extents that have been
@@ -762,11 +776,17 @@ phase6_func(
 		goto out_dbad;
 	}
 
+	ret = -bitmap_alloc(&vs.l_bad);
+	if (ret) {
+		str_liberror(ctx, ret, _("creating log badblock bitmap"));
+		goto out_rbad;
+	}
+
 	ret = read_verify_pool_alloc(ctx, XFS_DEV_DATA, remember_ioerr, &vs,
 			&vs.rvp_data);
 	if (ret) {
 		str_liberror(ctx, ret, _("creating datadev media verifier"));
-		goto out_rbad;
+		goto out_lbad;
 	}
 	if (ctx->fsinfo.fs_log) {
 		ret = read_verify_pool_alloc(ctx, XFS_DEV_LOG, remember_ioerr,
@@ -823,16 +843,17 @@ phase6_func(
 	 */
 	if (ret || ret2 || ret3) {
 		ret |= ret2 | ret3; /* caller only cares about non-zero/zero */
-		goto out_rbad;
+		goto out_lbad;
 	}
 	if (bitmap_empty(vs.d_bad) && !vs.d_trunc &&
 	    bitmap_empty(vs.r_bad) && !vs.r_trunc &&
-	    !vs.l_trunc)
-		goto out_rbad;
+	    bitmap_empty(vs.l_bad) && !vs.l_trunc)
+		goto out_lbad;
 
 	/* Scan the whole dir tree to see what matches the bad extents. */
 	ret = report_all_media_errors(ctx, &vs);
 
+	bitmap_free(&vs.l_bad);
 	bitmap_free(&vs.r_bad);
 	bitmap_free(&vs.d_bad);
 	return ret;
@@ -852,6 +873,8 @@ phase6_func(
 out_datapool:
 	read_verify_pool_abort(vs.rvp_data);
 	read_verify_pool_destroy(vs.rvp_data);
+out_lbad:
+	bitmap_free(&vs.l_bad);
 out_rbad:
 	bitmap_free(&vs.r_bad);
 out_dbad:
#55 lore raw

PATCH 17/22 xfs_scrub: index read-verify pools by xfs_device ids

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Refactor the read-verify pool array in struct media_verify_state so that we can index them via enum xfs_device. This will enable further cleanups in the next few patches.

---
 scrub/phase6.c |  103 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 55 insertions(+), 48 deletions(-)
scrub/phase6.c +55 -48
index c1ce222d4ce99f..a05bd7e1df1728 100644
@@ -42,9 +42,8 @@
 struct media_verify_state {
 	struct ptvar		*verify_schedules;
 
-	struct read_verify_pool	*rvp_data;
-	struct read_verify_pool	*rvp_log;
-	struct read_verify_pool	*rvp_realtime;
+	struct read_verify_pool	*rvp[XFS_DEV_RT + 1];
+
 	struct bitmap		*d_bad;		/* bytes */
 	struct bitmap		*r_bad;		/* bytes */
 	struct bitmap		*l_bad;		/* bytes */
@@ -53,28 +52,24 @@ struct media_verify_state {
 	bool			l_trunc:1;
 };
 
-/* Find the fd for a given device identifier. */
-static struct read_verify_pool *
-dev_to_pool(
-	struct scrub_ctx		*ctx,
-	struct media_verify_state	*vs,
-	dev_t				dev)
+/* Return XFS device index from fsmap device. */
+static enum xfs_device
+from_fsmap_dev(
+	struct scrub_ctx	*ctx,
+	dev_t			dev)
 {
 	if (ctx->mnt.fsgeom.rtstart) {
-		if (dev == XFS_DEV_DATA)
-			return vs->rvp_data;
-		if (dev == XFS_DEV_LOG)
-			return vs->rvp_log;
-		if (dev == XFS_DEV_RT)
-			return vs->rvp_realtime;
-	} else {
-		if (dev == ctx->fsinfo.fs_datadev)
-			return vs->rvp_data;
-		if (dev == ctx->fsinfo.fs_logdev)
-			return vs->rvp_log;
-		if (dev == ctx->fsinfo.fs_rtdev)
-			return vs->rvp_realtime;
+		if (dev < XFS_DEV_DATA || dev > XFS_DEV_RT)
+			abort();
+		return dev;
 	}
+
+	if (dev == ctx->fsinfo.fs_datadev)
+		return XFS_DEV_DATA;
+	if (dev == ctx->fsinfo.fs_logdev)
+		return XFS_DEV_LOG;
+	if (dev == ctx->fsinfo.fs_rtdev)
+		return XFS_DEV_RT;
 	abort();
 }
 
@@ -611,13 +606,12 @@ check_rmap(
 	void				*arg)
 {
 	struct media_verify_state	*vs = arg;
-	struct read_verify_pool		*rvp;
+	struct read_verify_pool		*rvp =
+		vs->rvp[from_fsmap_dev(ctx, map->fmr_device)];
 	struct read_verify_schedule	*rs;
 	bool				scheduled;
 	int				ret;
 
-	rvp = dev_to_pool(ctx, vs, map->fmr_device);
-
 	dbg_printf("rmap dev %d:%d phys %"PRIu64" owner %"PRId64
 			" offset %"PRIu64" len %"PRIu64" flags 0x%x\n",
 			major(map->fmr_device), minor(map->fmr_device),
@@ -686,11 +680,13 @@ force_one_verify(
 /* Wait for read/verify actions to finish, then return # bytes checked. */
 static int
 clean_pool(
-	struct read_verify_pool	*rvp,
-	unsigned long long	*bytes_checked)
+	struct media_verify_state	*vs,
+	enum xfs_device			dev,
+	unsigned long long		*bytes_checked)
 {
-	uint64_t		pool_checked;
-	int			ret;
+	struct read_verify_pool		*rvp = vs->rvp[dev];
+	uint64_t			pool_checked;
+	int				ret;
 
 	if (!rvp)
 		return 0;
@@ -749,6 +745,27 @@ remember_ioerr(
 		str_liberror(ctx, ret, _("setting bad block bitmap"));
 }
 
+static inline int
+alloc_pool(
+	struct scrub_ctx		*ctx,
+	struct media_verify_state	*vs,
+	enum xfs_device			dev)
+{
+	return read_verify_pool_alloc(ctx, dev, remember_ioerr, vs,
+			&vs->rvp[dev]);
+}
+
+static inline void
+free_pool(
+	struct media_verify_state	*vs,
+	enum xfs_device			dev)
+{
+	if (vs->rvp[dev]) {
+		read_verify_pool_abort(vs->rvp[dev]);
+		read_verify_pool_destroy(vs->rvp[dev]);
+	}
+}
+
 /*
  * Read verify all the file data blocks in a filesystem.  Since XFS doesn't
  * do data checksums, we trust that the underlying storage will pass back
@@ -782,15 +799,13 @@ phase6_func(
 		goto out_rbad;
 	}
 
-	ret = read_verify_pool_alloc(ctx, XFS_DEV_DATA, remember_ioerr, &vs,
-			&vs.rvp_data);
+	ret = alloc_pool(ctx, &vs, XFS_DEV_DATA);
 	if (ret) {
 		str_liberror(ctx, ret, _("creating datadev media verifier"));
 		goto out_lbad;
 	}
 	if (ctx->fsinfo.fs_log) {
-		ret = read_verify_pool_alloc(ctx, XFS_DEV_LOG, remember_ioerr,
-				&vs, &vs.rvp_log);
+		ret = alloc_pool(ctx, &vs, XFS_DEV_LOG);
 		if (ret) {
 			str_liberror(ctx, ret,
 					_("creating logdev media verifier"));
@@ -798,8 +813,7 @@ phase6_func(
 		}
 	}
 	if (ctx->fsinfo.fs_rt) {
-		ret = read_verify_pool_alloc(ctx, XFS_DEV_RT, remember_ioerr,
-				&vs, &vs.rvp_realtime);
+		ret = alloc_pool(ctx, &vs, XFS_DEV_RT);
 		if (ret) {
 			str_liberror(ctx, ret,
 					_("creating rtdev media verifier"));
@@ -825,15 +839,15 @@ phase6_func(
 	ptvar_free(vs.verify_schedules);
 	vs.verify_schedules = NULL;
 
-	ret = clean_pool(vs.rvp_data, &ctx->bytes_checked);
+	ret = clean_pool(&vs, XFS_DEV_DATA, &ctx->bytes_checked);
 	if (ret)
 		str_liberror(ctx, ret, _("flushing datadev verify pool"));
 
-	ret2 = clean_pool(vs.rvp_log, &ctx->bytes_checked);
+	ret2 = clean_pool(&vs, XFS_DEV_LOG, &ctx->bytes_checked);
 	if (ret2)
 		str_liberror(ctx, ret2, _("flushing logdev verify pool"));
 
-	ret3 = clean_pool(vs.rvp_realtime, &ctx->bytes_checked);
+	ret3 = clean_pool(&vs, XFS_DEV_RT, &ctx->bytes_checked);
 	if (ret3)
 		str_liberror(ctx, ret3, _("flushing rtdev verify pool"));
 
@@ -861,18 +875,11 @@ phase6_func(
 out_schedules:
 	ptvar_free(vs.verify_schedules);
 out_rtpool:
-	if (vs.rvp_realtime) {
-		read_verify_pool_abort(vs.rvp_realtime);
-		read_verify_pool_destroy(vs.rvp_realtime);
-	}
+	free_pool(&vs, XFS_DEV_RT);
 out_logpool:
-	if (vs.rvp_log) {
-		read_verify_pool_abort(vs.rvp_log);
-		read_verify_pool_destroy(vs.rvp_log);
-	}
+	free_pool(&vs, XFS_DEV_LOG);
 out_datapool:
-	read_verify_pool_abort(vs.rvp_data);
-	read_verify_pool_destroy(vs.rvp_data);
+	free_pool(&vs, XFS_DEV_DATA);
 out_lbad:
 	bitmap_free(&vs.l_bad);
 out_rbad:
#57 lore raw

PATCH 18/22 xfs_scrub: move failmap and other outputs into read_verify_pool

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Remove all the indirect verify IO error function calls and whatnot by making the read_verify_pool track the ranges of failed media and other problems. Add some new helper functions that report on the outcome of the read verifciation so that phase6 can report on what happened.

---
 scrub/read_verify.h |   13 +++-
 scrub/phase6.c      |  184 +++++++++++++--------------------------------------
 scrub/read_verify.c |  129 +++++++++++++++++++++++++++++++-----
 3 files changed, 170 insertions(+), 156 deletions(-)
scrub/read_verify.h +11 -2
index 6a338abb896bc3..c46ba1e76a8d03 100644
@@ -21,7 +21,6 @@ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx,
 		int error, void *arg);
 
 int read_verify_pool_alloc(struct scrub_ctx *ctx, enum xfs_device dev,
-		read_verify_ioerr_fn_t ioerr_fn, void *ioerr_arg,
 		struct read_verify_pool **prvp);
 void read_verify_pool_abort(struct read_verify_pool *rvp);
 int read_verify_pool_flush(struct read_verify_pool *rvp);
@@ -31,7 +30,17 @@ int read_verify_schedule_now(struct read_verify_schedule *rs);
 bool try_read_verify_schedule_io(struct read_verify_schedule *rs,
 		struct read_verify_pool *rvp, uint64_t start, uint64_t length);
 
-int read_verify_bytes(struct read_verify_pool *rvp, uint64_t *bytes);
+bool read_verify_ok(const struct read_verify_pool *rvp);
+bool read_verify_truncated(const struct read_verify_pool *rvp);
+uint64_t read_verify_progress(const struct read_verify_pool *rvp);
+
+int read_verify_iterate_failed(struct read_verify_pool *rvp,
+		int (*fn)(uint64_t, uint64_t, void *), void *arg);
+int read_verify_iterate_failed_range(struct read_verify_pool *rvp,
+		uint64_t start, uint64_t length,
+		int (*fn)(uint64_t, uint64_t, void *), void *arg);
+bool read_verify_has_failed(struct read_verify_pool *rvp, uint64_t start,
+		uint64_t len);
 
 unsigned int read_verify_nproc(struct scrub_ctx *ctx);
 
scrub/phase6.c +47 -137

311 lines not on this page · raw message

scrub/read_verify.c +112 -17

205 lines not on this page · raw message

#59 lore raw

PATCH 19/22 xfs_scrub: clean up device-related error messages

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use consistent terminology for the data/log/rt device in the error messages.

---
 scrub/phase6.c |   16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
scrub/phase6.c +8 -8
index bf9ef3d12690ab..14f1a26ab4ff7f 100644
@@ -551,7 +551,7 @@ report_all_media_errors(
 
 	ret = report_disk_ioerrs(ctx, vs, XFS_DEV_DATA);
 	if (ret) {
-		str_liberror(ctx, ret, _("walking datadev io errors"));
+		str_liberror(ctx, ret, _("walking data device io errors"));
 		return ret;
 	}
 
@@ -563,7 +563,7 @@ report_all_media_errors(
 
 	ret = report_disk_ioerrs(ctx, vs, XFS_DEV_RT);
 	if (ret) {
-		str_liberror(ctx, ret, _("walking rtdev io errors"));
+		str_liberror(ctx, ret, _("walking rt device io errors"));
 		return ret;
 	}
 
@@ -725,14 +725,14 @@ phase6_func(
 
 	ret = alloc_pool(ctx, &vs, XFS_DEV_DATA);
 	if (ret) {
-		str_liberror(ctx, ret, _("creating datadev media verifier"));
+		str_liberror(ctx, ret, _("creating data device media verifier"));
 		return ret;
 	}
 	if (ctx->fsinfo.fs_log) {
 		ret = alloc_pool(ctx, &vs, XFS_DEV_LOG);
 		if (ret) {
 			str_liberror(ctx, ret,
-					_("creating logdev media verifier"));
+					_("creating log device media verifier"));
 			goto out_datapool;
 		}
 	}
@@ -740,7 +740,7 @@ phase6_func(
 		ret = alloc_pool(ctx, &vs, XFS_DEV_RT);
 		if (ret) {
 			str_liberror(ctx, ret,
-					_("creating rtdev media verifier"));
+					_("creating rt device media verifier"));
 			goto out_logpool;
 		}
 	}
@@ -765,15 +765,15 @@ phase6_func(
 
 	ret = clean_pool(&vs, XFS_DEV_DATA, &ctx->bytes_checked, &ok);
 	if (ret)
-		str_liberror(ctx, ret, _("flushing datadev verify pool"));
+		str_liberror(ctx, ret, _("flushing data device verify pool"));
 
 	ret2 = clean_pool(&vs, XFS_DEV_LOG, &ctx->bytes_checked, &ok);
 	if (ret2)
-		str_liberror(ctx, ret2, _("flushing logdev verify pool"));
+		str_liberror(ctx, ret2, _("flushing log device verify pool"));
 
 	ret3 = clean_pool(&vs, XFS_DEV_RT, &ctx->bytes_checked, &ok);
 	if (ret3)
-		str_liberror(ctx, ret3, _("flushing rtdev verify pool"));
+		str_liberror(ctx, ret3, _("flushing rt device verify pool"));
 
 	/*
 	 * If the verify flush didn't work or we found no bad blocks, we're
#61 lore raw

PATCH 20/22 xfs_scrub: drop SCSI_VERIFY code from disk.

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that we have a media verification ioctl in the kernel, drop the SCSI_VERIFY code, which enables us to drop the dependency on sg and obviates the need to fix some unit-handling bugs in the HDIO_GETGEO code.

A subsequent patch will enable larger verification IO sizes, which this old code cannot handle.

---
 scrub/disk.h            |    2 -
 doc/README-env-vars.txt |    1 
 scrub/disk.c            |  133 -----------------------------------------------
 scrub/read_verify.c     |    5 --
 scrub/xfs_scrub.c       |    1 
 5 files changed, 1 insertion(+), 141 deletions(-)
scrub/disk.h +0 -2
index 73c73ab57fb5c7..a9155bf1f75eb4 100644
@@ -6,7 +6,6 @@
 #ifndef XFS_SCRUB_DISK_H_
 #define XFS_SCRUB_DISK_H_
 
-#define DISK_FLAG_SCSI_VERIFY	0x1
 struct disk {
 	struct stat	d_sb;
 	int		d_fd;
@@ -15,7 +14,6 @@ struct disk {
 	unsigned int	d_flags;
 	unsigned int	d_blksize;	/* bytes */
 	uint64_t	d_size;		/* bytes */
-	uint64_t	d_start;	/* bytes */
 };
 
 unsigned int disk_heads(struct disk *disk);
doc/README-env-vars.txt +0 -1
index eec59a82513f36..6862dcad71e217 100644
@@ -17,7 +17,6 @@ Known debug tweaks (pass -d and set the environment variable):
 XFS_SCRUB_FORCE_ERROR        -- pretend all metadata is corrupt
 XFS_SCRUB_FORCE_REPAIR       -- repair all metadata even if it's ok
 XFS_SCRUB_NO_KERNEL          -- pretend there is no kernel ioctl
-XFS_SCRUB_NO_SCSI_VERIFY     -- disable SCSI VERIFY (if present)
 XFS_SCRUB_PHASE              -- run only this scrub phase
 XFS_SCRUB_THREADS            -- start exactly this number of threads
 
scrub/disk.c +0 -133

173 lines not on this page · raw message

scrub/read_verify.c +1 -4
index 4eb7484317e5ca..6789f5f668b46c 100644
@@ -27,10 +27,7 @@
  * pool takes care of issuing multiple IOs to the device, if possible.
  */
 
-/*
- * Perform all IO in 32M chunks.  This cannot exceed 65536 sectors
- * because that's the biggest SCSI VERIFY(16) we dare to send.
- */
+/* Perform all verification IO in 32M chunks. */
 #define RVP_IO_MAX_SIZE		(33554432)
 
 /*
scrub/xfs_scrub.c +0 -1
index b74dc1635141aa..aae24ec83c1c75 100644
@@ -111,7 +111,6 @@
  * XFS_SCRUB_FORCE_ERROR	-- pretend all metadata is corrupt
  * XFS_SCRUB_FORCE_REPAIR	-- repair all metadata even if it's ok
  * XFS_SCRUB_NO_KERNEL		-- pretend there is no kernel ioctl
- * XFS_SCRUB_NO_SCSI_VERIFY	-- disable SCSI VERIFY (if present)
  * XFS_SCRUB_PHASE		-- run only this scrub phase
  * XFS_SCRUB_THREADS		-- start exactly this number of threads
  * XFS_SCRUB_DISK_ERROR_INTERVAL-- simulate a disk error every this many bytes
Reviewed-by on v1 20/22
#62 lore raw

Re: [PATCH 20/22] xfs_scrub: drop SCSI_VERIFY code from disk.

Quoted 9 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that we have a media verification ioctl in the kernel, drop the SCSI_VERIFY code, which enables us to drop the dependency on sg and obviates the need to fix some unit-handling bugs in the HDIO_GETGEO code.

A subsequent patch will enable larger verification IO sizes, which this old code cannot handle.

Good riddance!

#63 lore raw

PATCH 21/22 xfs_scrub: raise media verification IO limits

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

To avoid starving other threads of disk IO resources, the read-verify pool limits the size of verification IOs to limit bandwidth consumption, and it is willing to over-verify some amount of unwritten media to reduce the number of verification IO requests sent to the device.

However, these limits were set in 2018 when areal densities were lower and disk bandwidth was more limited. Increase them now to reduce scan time on the author's system by 10% in foreground and 50% in background mode.

---
 scrub/read_verify.c |   31 +++++++++++++++++++++++--------
 1 file changed, 23 insertions(+), 8 deletions(-)
scrub/read_verify.c +23 -8
index 6789f5f668b46c..01f9134799384f 100644
@@ -10,6 +10,7 @@
 #include "libfrog/workqueue.h"
 #include "libfrog/paths.h"
 #include "libfrog/bitmap.h"
+#include "libfrog/convert.h"
 #include "xfs_scrub.h"
 #include "common.h"
 #include "counter.h"
@@ -28,23 +29,36 @@
  */
 
 /* Perform all verification IO in 32M chunks. */
-#define RVP_IO_MAX_SIZE		(33554432)
+#define RVP_IO_MAX_SIZE			MEGABYTES(32)
 
 /*
- * If we're running in the background then we perform IO in 128k chunks
+ * If we're running in the background then we perform IO in 256k chunks
  * to reduce the load on the IO subsystem.
  */
-#define RVP_BACKGROUND_IO_MAX_SIZE	(131072)
+#define RVP_BG_IO_MAX_SIZE		KILOBYTES(256)
 
 /* What's the real maximum IO size? */
 static inline unsigned int
 rvp_io_max_size(void)
 {
-	return bg_mode > 0 ? RVP_BACKGROUND_IO_MAX_SIZE : RVP_IO_MAX_SIZE;
+	return bg_mode > 0 ? RVP_BG_IO_MAX_SIZE : RVP_IO_MAX_SIZE;
 }
 
-/* Tolerate 64k holes in adjacent read verify requests. */
-#define RVP_IO_BATCH_LOCALITY	(65536)
+/* Tolerate 2M holes in adjacent read verify requests. */
+#define RVP_IO_BATCH_LOCALITY		MEGABYTES(2)
+
+/*
+ * Tolerate 256k holes in adjacent read verify requests when running in the
+ * background.
+ */
+#define RVP_BG_IO_BATCH_LOCALITY	KILOBYTES(256)
+
+/* How many holes are we willing to verify to reduce IO count? */
+static inline unsigned int
+rvp_io_batch_locality(void)
+{
+	return bg_mode > 0 ? RVP_BG_IO_BATCH_LOCALITY : RVP_IO_BATCH_LOCALITY;
+}
 
 struct read_verify {
 	uint64_t		io_start;	/* bytes */
@@ -488,6 +502,7 @@ try_read_verify_schedule_io(
 {
 	uint64_t			req_end;
 	uint64_t			rv_end;
+	const unsigned int		locality = rvp_io_batch_locality();
 
 	assert(rvp->readbuf);
 
@@ -513,9 +528,9 @@ try_read_verify_schedule_io(
 	 * we can combine them.
 	 */
 	if (rs->rvp == rvp && rs->io_length > 0 &&
-	    ((start >= rs->io_start && start <= rv_end + RVP_IO_BATCH_LOCALITY) ||
+	    ((start >= rs->io_start && start <= rv_end + locality) ||
 	     (rs->io_start >= start &&
-	      rs->io_start <= req_end + RVP_IO_BATCH_LOCALITY))) {
+	      rs->io_start <= req_end + locality))) {
 		rs->io_start = min(rs->io_start, start);
 		rs->io_length = max(req_end, rv_end) - rs->io_start;
 
Reviewed-by on v1 21/22
#64 lore raw

Re: [PATCH 21/22] xfs_scrub: raise media verification IO limits

Quoted 11 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

To avoid starving other threads of disk IO resources, the read-verify pool limits the size of verification IOs to limit bandwidth consumption, and it is willing to over-verify some amount of unwritten media to reduce the number of verification IO requests sent to the device.

However, these limits were set in 2018 when areal densities were lower and disk bandwidth was more limited. Increase them now to reduce scan time on the author's system by 10% in foreground and 50% in background mode.

This looks good for now. At some point we'll need to return to fine-tune this better for different media.

#65 lore raw

Re: [PATCH 21/22] xfs_scrub: raise media verification IO limits

Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

To avoid starving other threads of disk IO resources, the read-verify pool limits the size of verification IOs to limit bandwidth consumption, and it is willing to over-verify some amount of unwritten media to reduce the number of verification IO requests sent to the device.

However, these limits were set in 2018 when areal densities were lower and disk bandwidth was more limited. Increase them now to reduce scan time on the author's system by 10% in foreground and 50% in background mode.

Quoted 4 lines

This looks good for now. At some point we'll need to return to fine-tune this better for different media.

<nod> Thanks for reviewing and sharing patches! :)

--D

#66 lore raw

PATCH 22/22 xfs_scrub: allow overrides of the media verification IO limits

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Allow power users to override the media verification IO size limits via magic environment variables. For the background service, this can be done via:

[Service]
Environment=XFS_SCRUB_VERIFY_MAX_SIZE=128M

---
 doc/README-env-vars.txt |    3 ++
 scrub/read_verify.c     |   62 +++++++++++++++++++++++++++++++++++++++--------
 scrub/xfs_scrub.c       |    3 ++
 3 files changed, 57 insertions(+), 11 deletions(-)
doc/README-env-vars.txt +3 -0
index 6862dcad71e217..03faf0882f1ae0 100644
@@ -23,3 +23,6 @@ XFS_SCRUB_THREADS            -- start exactly this number of threads
 Available even in non-debug mode:
 SERVICE_MODE                 -- compress all error codes to 1 for LSB
                                 service action compliance
+XFS_SCRUB_VERIFY_MAX_SIZE    -- maximum size of media verification requests
+XFS_SCRUB_VERIFY_BATCH_LOCALITY -- coalesce sparse areas of up to this size
+                                   to batch verification requests
scrub/read_verify.c +51 -11

102 lines not on this page · raw message

scrub/xfs_scrub.c +3 -0
index aae24ec83c1c75..0faf49b862b932 100644
@@ -120,6 +120,9 @@
  * Available even in non-debug mode:
  * SERVICE_MODE			-- compress all error codes to 1 for LSB
  *				   service action compliance
+ * XFS_SCRUB_VERIFY_MAX_SIZE    -- maximum size of media verification requests
+ * XFS_SCRUB_VERIFY_BATCH_LOCALITY -- coalesce sparse areas of up to this size
+ *                                    to batch verification requests
  */
 
 /* Program name; needed for libfrog error reports. */
#67 lore raw

Re: [PATCH 22/22] xfs_scrub: allow overrides of the media verification IO limits

Quoted 8 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Allow power users to override the media verification IO size limits via magic environment variables. For the background service, this can be done via:

[Service]
Environment=XFS_SCRUB_VERIFY_MAX_SIZE=128M

So you'll need to hack the systemd unit files? How could we set this on a per-file system basis?

Not really arguing against this, but we might end up needing more flexbility in the end.

#68 lore raw

Re: [PATCH 22/22] xfs_scrub: allow overrides of the media verification IO limits

Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Allow power users to override the media verification IO size limits via magic environment variables. For the background service, this can be done via:

Quoted 8 lines
Quoted 2 lines

[Service]
Environment=XFS_SCRUB_VERIFY_MAX_SIZE=128M

So you'll need to hack the systemd unit files? How could we set this on a per-file system basis?

Not really arguing against this, but we might end up needing more flexbility in the end.

I'd do per-fs tweaks by defining an xfs_property and telling users to set it, e.g.

# xfs_property /home set scrub_verify_max_size=128M

I don't think we need to define the property right now, that can wait until someone has time to do a more in depth analysis of what settings adjustments are needed for modern hardware. I'm keener on figuring out something that'd work more automagically because sysadmins are lazy. :)

--D

#69 lore raw

Re: [PATCH 22/22] xfs_scrub: allow overrides of the media verification IO limits

Quoted 10 lines · Darrick J. Wong
Quoted 5 lines

So you'll need to hack the systemd unit files? How could we set this on a per-file system basis?

Not really arguing against this, but we might end up needing more flexbility in the end.

I'd do per-fs tweaks by defining an xfs_property and telling users to set it, e.g.

# xfs_property /home set scrub_verify_max_size=128M

Sounds reasonable.

Quoted 4 lines

I don't think we need to define the property right now, that can wait until someone has time to do a more in depth analysis of what settings adjustments are needed for modern hardware. I'm keener on figuring out something that'd work more automagically because sysadmins are lazy. :)

Agreed. Another thing we might want is to record the scrub progress somewhere so that we don't always start from the beginning when interrupted. Also not really needed now, I'd rather land the code first.

#70 lore raw

Re: [PATCH 22/22] xfs_scrub: allow overrides of the media verification IO limits

Quoted 19 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 5 lines

So you'll need to hack the systemd unit files? How could we set this on a per-file system basis?

Not really arguing against this, but we might end up needing more flexbility in the end.

I'd do per-fs tweaks by defining an xfs_property and telling users to set it, e.g.

# xfs_property /home set scrub_verify_max_size=128M

Sounds reasonable.

Quoted 4 lines

I don't think we need to define the property right now, that can wait until someone has time to do a more in depth analysis of what settings adjustments are needed for modern hardware. I'm keener on figuring out something that'd work more automagically because sysadmins are lazy. :)

Quoted 4 lines

Agreed. Another thing we might want is to record the scrub progress somewhere so that we don't always start from the beginning when interrupted. Also not really needed now, I'd rather land the code first.

<nod> I'll send out the most recent libxfs-7.0 resync patches shortly, so Andrey can get started on that.

--D