Skip to contentSkip to conversation

xfsprogs: autonomous self healing of filesystems

PATCH Superseded Darrick J. Wong 6 participants 111 messages Mar 3 – Mar 9

Subsystems (3)

v8 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

PATCH v8 1/1 xfsprogs: autonomous self healing of filesystems

Hi all,

This patchset contains the userspace and QA changes (xfs_healer) needed to put to use all the new kernel functionality to deliver live information about filesystem health events (xfs_healthmon.c) to userspace.

In userspace, we create a new daemon program that will read the event objects and initiate repairs automatically. This daemon is managed entirely by systemd and will not block unmounting of the filesystem unless repairs are ongoing. They are auto-started by a starter service that uses fanotify.

When the patchsets under this cover letter are merged, online fsck for XFS will at long last be fully feature complete. The passive scan parts have been done since mid-2024, this final part adds proactive repair.

v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
and filter bio status flags better
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status

--D

#1 lore raw

PATCH v8 1/1 xfsprogs: autonomous self healing of filesystems

Hi all,

This patchset builds new functionality to deliver live information about filesystem health events to userspace. This is done by creating an anonymous file that can be read() for events by userspace programs. Events are captured by hooking various parts of XFS and iomap so that metadata health failures, file I/O errors, and major changes in filesystem state (unmounts, shutdowns, etc.) can be observed by programs.

When an event occurs, the hook functions queue an event object to each event anonfd for later processing. Programs must have CAP_SYS_ADMIN to open the anonfd and there's a maximum event lag to prevent resource overconsumption. The events themselves can be read() from the anonfd as C structs for the xfs_healer daemon.

In userspace, we create a new daemon program that will read the event objects and initiate repairs automatically. This daemon is managed entirely by systemd and will not block unmounting of the filesystem unless repairs are ongoing. They are auto-started by a starter service that uses fanotify.

v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=health-monitoring

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=health-monitoring

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=health-monitoring
---
Commits in this patchset:
* libfrog: add a function to grab the path from an open fd and a file handle
* libfrog: create healthmon event log library functions
* libfrog: add support code for starting systemd services programmatically
* libfrog: hoist a couple of service helper functions
* man2: document the healthmon ioctl
* man2: document the media verification ioctl
* xfs_io: monitor filesystem health events
* xfs_io: add a media verify command
* xfs_healer: create daemon to listen for health events
* xfs_healer: enable repairing filesystems
* xfs_healer: use getparents to look up file names
* xfs_healer: create a per-mount background monitoring service
* xfs_healer: create a service to start the per-mount healer service
* xfs_healer: don't start service if kernel support unavailable
* xfs_healer: use the autofsck fsproperty to select mode
* xfs_healer: run full scrub after lost corruption events or targeted repair failure
* xfs_healer: use getmntent to find moved filesystems
* xfs_healer: validate that repair fds point to the monitored fs
* xfs_healer: add a manual page
* xfs_scrub: use the verify media ioctl during phase 6 if possible
* xfs_scrub: perform media scanning of the log region
* xfs_io: add listmount command
* xfs_io: print systemd service names
* mkfs: enable online repair if all backrefs are enabled
* debian: enable xfs_healer on the root filesystem by default
* debian/control: listify the build dependencies

---
 healer/xfs_healer.h                            |   88 +++
 io/io.h                                        |    8 
 libfrog/flagmap.h                              |   23 +
 libfrog/fsproperties.h                         |    5 
 libfrog/getparents.h                           |    4 
 libfrog/healthevent.h                          |   55 ++
 libfrog/systemd.h                              |   55 ++
 scrub/disk.h                                   |   11 
 Makefile                                       |    5 
 configure.ac                                   |   12 
 debian/control                                 |   14 +
 debian/postinst                                |    8 
 debian/prerm                                   |   13 +
 debian/rules                                   |    3 
 healer/Makefile                                |   70 +++
 healer/fsrepair.c                              |  342 ++++++++++++++
 healer/system-xfs_healer.slice                 |   31 +
 healer/weakhandle.c                            |  266 +++++++++++
 healer/xfs_healer.c                            |  605 ++++++++++++++++++++++++
 healer/[email protected]                  |  108 ++++
 healer/xfs_healer_start.c                      |  372 +++++++++++++++
 healer/xfs_healer_start.service.in             |   85 +++
 include/builddefs.in                           |   12 
 io/Makefile                                    |   16 +
 io/healthmon.c                                 |  186 +++++++
 io/init.c                                      |    3 
 io/listmount.c                                 |  383 +++++++++++++++
 io/scrub.c                                     |   75 +++
 io/verify_media.c                              |  180 +++++++
 libfrog/Makefile                               |   10 
 libfrog/flagmap.c                              |   79 +++
 libfrog/getparents.c                           |   93 +++-
 libfrog/healthevent.c                          |  477 +++++++++++++++++++
 libfrog/systemd.c                              |  181 +++++++
 m4/package_libcdev.m4                          |   97 ++++
 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 |   75 +++
 man/man2/ioctl_xfs_health_monitor.2            |  464 ++++++++++++++++++
 man/man2/ioctl_xfs_verify_media.2              |  185 +++++++
 man/man8/Makefile                              |   40 +-
 man/man8/xfs_healer.8                          |  109 ++++
 man/man8/xfs_healer_start.8                    |   37 +
 man/man8/xfs_io.8                              |  134 +++++
 mkfs/xfs_mkfs.c                                |    9 
 scrub/Makefile                                 |   13 -
 scrub/disk.c                                   |   40 ++
 scrub/phase1.c                                 |   25 +
 scrub/phase6.c                                 |   11 
 scrub/read_verify.c                            |    2 
 scrub/xfs_scrub.c                              |   32 -
 49 files changed, 5090 insertions(+), 61 deletions(-)
 create mode 100644 healer/xfs_healer.h
 create mode 100644 libfrog/flagmap.h
 create mode 100644 libfrog/healthevent.h
 create mode 100644 libfrog/systemd.h
 create mode 100644 debian/prerm
 create mode 100644 healer/Makefile
 create mode 100644 healer/fsrepair.c
 create mode 100644 healer/system-xfs_healer.slice
 create mode 100644 healer/weakhandle.c
 create mode 100644 healer/xfs_healer.c
 create mode 100644 healer/[email protected]
 create mode 100644 healer/xfs_healer_start.c
 create mode 100644 healer/xfs_healer_start.service.in
 create mode 100644 io/healthmon.c
 create mode 100644 io/listmount.c
 create mode 100644 io/verify_media.c
 create mode 100644 libfrog/flagmap.c
 create mode 100644 libfrog/healthevent.c
 create mode 100644 libfrog/systemd.c
 create mode 100644 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2
 create mode 100644 man/man2/ioctl_xfs_health_monitor.2
 create mode 100644 man/man2/ioctl_xfs_verify_media.2
 create mode 100644 man/man8/xfs_healer.8
 create mode 100644 man/man8/xfs_healer_start.8
#2 lore raw

PATCH 1/26 libfrog: add a function to grab the path from an open fd and a file handle

From: Darrick J. Wong <[email protected]>

handle_walk_paths operates on a file handle, but requires that the fs has been registered with libhandle via path_to_fshandle. For a normal libhandle client this is the desirable behavior because the application *should* maintain an open fd to the filesystem mount.

However for xfs_healer this isn't going to work well because the healer mustn't pin the mount while it's running. It's smart enough to know how to find and reconnect to the mountpoint, but libhandle doesn't have any such concept.

Therefore, alter the libfrog getparents code so that xfs_healer can pass in the mountpoint and reconnected fd without needing libhandle. All we're really doing here is trying to obtain a user-visible path for a file that encountered problems for logging purposes; if it fails, we'll fall back to logging the inode number.

---
 libfrog/getparents.h |    4 ++
 libfrog/getparents.c |   93 ++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 82 insertions(+), 15 deletions(-)
libfrog/getparents.h +4 -0
index 8098d594219b4c..e1df30889c7606 100644
@@ -39,4 +39,8 @@ int fd_to_path(int fd, size_t ioctl_bufsize, char *path, size_t pathlen);
 int handle_to_path(const void *hanp, size_t hlen, size_t ioctl_bufsize,
 		char *path, size_t pathlen);
 
+int handle_walk_paths_fd(const char *mntpt, int mntfd, const void *hanp,
+		size_t hanlen, size_t ioctl_bufsize, walk_path_fn fn,
+		void *arg);
+
 #endif /* __LIBFROG_GETPARENTS_H_ */
libfrog/getparents.c +78 -15

141 lines not on this page · raw message

#4 lore raw

PATCH 2/26 libfrog: create healthmon event log library functions

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some helper functions to log health monitoring events so that xfs_io and xfs_healer can share logging code.

---
 libfrog/flagmap.h     |   20 +++
 libfrog/healthevent.h |   43 ++++++
 libfrog/Makefile      |    4 +
 libfrog/flagmap.c     |   62 ++++++++
 libfrog/healthevent.c |  360 +++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 489 insertions(+)
 create mode 100644 libfrog/flagmap.h
 create mode 100644 libfrog/healthevent.h
 create mode 100644 libfrog/flagmap.c
 create mode 100644 libfrog/healthevent.c
libfrog/flagmap.h new file +20 -0
new file mode 100644
index 00000000000000..8031d75a7c02a8
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025-2026 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef LIBFROG_FLAGMAP_H_
+#define LIBFROG_FLAGMAP_H_
+
+struct flag_map {
+	unsigned long long	flag;
+	const char		*string;
+};
+
+void mask_to_string(const struct flag_map *map, unsigned long long mask,
+		const char *delimiter, char *buf, size_t bufsize);
+
+const char *value_to_string(const struct flag_map *map,
+		unsigned long long value);
+
+#endif /* LIBFROG_FLAGMAP_H_ */
libfrog/healthevent.h new file +43 -0

43 lines not on this page · raw message

libfrog/Makefile +4 -0
index 927bd8d0957fab..bccd9289e5dd79 100644
@@ -19,11 +19,13 @@ bulkstat.c \
 convert.c \
 crc32.c \
 file_exchange.c \
+flagmap.c \
 fsgeom.c \
 fsproperties.c \
 fsprops.c \
 getparents.c \
 histogram.c \
+healthevent.c \
 file_attr.c \
 list_sort.c \
 linux.c \
@@ -51,11 +53,13 @@ dahashselftest.h \
 div64.h \
 fakelibattr.h \
 file_exchange.h \
+flagmap.h \
 fsgeom.h \
 fsproperties.h \
 fsprops.h \
 getparents.h \
 handle_priv.h \
+healthevent.h \
 histogram.h \
 file_attr.h \
 logging.h \
libfrog/flagmap.c new file +62 -0

62 lines not on this page · raw message

libfrog/healthevent.c new file +360 -0

360 lines not on this page · raw message

#6 lore raw

PATCH 3/26 libfrog: add support code for starting systemd services programmatically

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some simple routines for computing the name of systemd service instances and starting systemd services. These will be used by the xfs_healer_start service to start per-filesystem xfs_healer service instances.

Note that we run systemd helper programs as subprocesses for a couple of reasons. First, the path-escaping functionality is not a part of any library-accessible API, which means it can only be accessed via systemd-escape(1). Second, although the service startup functionality can be reached via dbus, doing so would introduce a new library dependency. Systemd is also undergoing a dbus -> varlink RPC transition so we avoid that mess by calling the cli systemctl(1) program.

---
 libfrog/systemd.h     |   20 +++++
 configure.ac          |    1 
 include/builddefs.in  |    1 
 libfrog/Makefile      |    6 ++
 libfrog/systemd.c     |  181 +++++++++++++++++++++++++++++++++++++++++++++++++
 m4/package_libcdev.m4 |   19 +++++
 6 files changed, 228 insertions(+)
 create mode 100644 libfrog/systemd.h
 create mode 100644 libfrog/systemd.c
libfrog/systemd.h new file +20 -0
new file mode 100644
index 00000000000000..4f414bc3c1e9c3
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2026 Oracle.  All rights reserved.
+ * All Rights Reserved.
+ */
+#ifndef __LIBFROG_SYSTEMD_H__
+#define __LIBFROG_SYSTEMD_H__
+
+int systemd_path_instance_unit_name(const char *unit_template,
+		const char *path, char *unitname, size_t unitnamelen);
+
+enum systemd_unit_manage {
+	UM_STOP,
+	UM_START,
+	UM_RESTART,
+};
+
+int systemd_manage_unit(enum systemd_unit_manage how, const char *unitname);
+
+#endif /* __LIBFROG_SYSTEMD_H__ */
configure.ac +1 -0
index a8b8f7d5066fb6..8d2bbb9ef88bb9 100644
@@ -182,6 +182,7 @@ AC_CONFIG_UDEV_RULE_DIR
 AC_HAVE_BLKID_TOPO
 AC_HAVE_TRIVIAL_AUTO_VAR_INIT
 AC_STRERROR_R_RETURNS_STRING
+AC_HAVE_CLOSE_RANGE
 
 if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then
         AC_PACKAGE_CHECK_UBSAN
include/builddefs.in +1 -0
index b38a099b7d525a..4a2cb757c0bdb3 100644
@@ -118,6 +118,7 @@ HAVE_UDEV = @have_udev@
 UDEV_RULE_DIR = @udev_rule_dir@
 HAVE_LIBURCU_ATOMIC64 = @have_liburcu_atomic64@
 STRERROR_R_RETURNS_STRING = @strerror_r_returns_string@
+HAVE_CLOSE_RANGE = @have_close_range@
 
 GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall
 #	   -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
libfrog/Makefile +6 -0
index bccd9289e5dd79..89a0332ae85372 100644
@@ -36,6 +36,7 @@ ptvar.c \
 radix-tree.c \
 randbytes.c \
 scrub.c \
+systemd.c \
 util.c \
 workqueue.c \
 zones.c
@@ -70,6 +71,7 @@ radix-tree.h \
 randbytes.h \
 scrub.h \
 statx.h \
+systemd.h \
 workqueue.h \
 zones.h
 
@@ -90,6 +92,10 @@ ifeq ($(HAVE_GETRANDOM_NONBLOCK),yes)
 LCFLAGS += -DHAVE_GETRANDOM_NONBLOCK
 endif
 
+ifeq ($(HAVE_CLOSE_RANGE),yes)
+CFLAGS += -DHAVE_CLOSE_RANGE
+endif
+
 default: ltdepend $(LTLIBRARY) $(GETTEXT_PY)
 
 crc32table.h: gen_crc32table.c crc32defs.h
libfrog/systemd.c new file +181 -0

181 lines not on this page · raw message

m4/package_libcdev.m4 +19 -0
index c5538c30d2518a..b3d87229d3367a 100644
@@ -347,3 +347,22 @@ puts(strerror_r(0, buf, sizeof(buf)));
     CFLAGS="$OLD_CFLAGS"
     AC_SUBST(strerror_r_returns_string)
   ])
+
+#
+# Check if close_range exists
+#
+AC_DEFUN([AC_HAVE_CLOSE_RANGE],
+  [AC_MSG_CHECKING([for close_range])
+    AC_LINK_IFELSE(
+    [AC_LANG_PROGRAM([[
+#define _GNU_SOURCE
+#include <unistd.h>
+#include <linux/close_range.h>
+  ]], [[
+close_range(0, 0, 0);
+  ]])
+    ], have_close_range=yes
+       AC_MSG_RESULT(yes),
+       AC_MSG_RESULT(no))
+    AC_SUBST(have_close_range)
+  ])
#7 lore raw

Re: [PATCH 03/26] libfrog: add support code for starting systemd services programmatically

Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some simple routines for computing the name of systemd service instances and starting systemd services. These will be used by the xfs_healer_start service to start per-filesystem xfs_healer service instances.

Quoted 7 lines

Note that we run systemd helper programs as subprocesses for a couple of reasons. First, the path-escaping functionality is not a part of any library-accessible API, which means it can only be accessed via systemd-escape(1). Second, although the service startup functionality can be reached via dbus, doing so would introduce a new library dependency. Systemd is also undergoing a dbus -> varlink RPC transition so we avoid that mess by calling the cli systemctl(1) program.

Just curious: did you run this past the systemd folks? Shelling out always feel a bit iffy, and they're usually happy to help on how to integrate with their services, so just asking might result in a better way.

#8 lore raw

Re: [PATCH 03/26] libfrog: add support code for starting systemd services programmatically

Quoted 16 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some simple routines for computing the name of systemd service instances and starting systemd services. These will be used by the xfs_healer_start service to start per-filesystem xfs_healer service instances.

Quoted 7 lines

Note that we run systemd helper programs as subprocesses for a couple of reasons. First, the path-escaping functionality is not a part of any library-accessible API, which means it can only be accessed via systemd-escape(1). Second, although the service startup functionality can be reached via dbus, doing so would introduce a new library dependency. Systemd is also undergoing a dbus -> varlink RPC transition so we avoid that mess by calling the cli systemctl(1) program.

Quoted 4 lines

Just curious: did you run this past the systemd folks? Shelling out always feel a bit iffy, and they're usually happy to help on how to integrate with their services, so just asking might result in a better way.

I'll do that, though even if they add a dbus/varlink endpoint for path escaping, it'll be a few releases more until we can depend on it existing in the distros. :(

Service startup can be done fairly easily with something like this, though obviously you'd want to do real error checking here:

static DBusConnection*
connect_to_system_bus(void)

{
	// Connect to the system bus (requires root or polkit permissions)
	DBusConnection *conn = dbus_bus_get(DBUS_BUS_SYSTEM, error);

	return conn;
}

int
systemd_stop_service(
const char *service_name)

{
	DBusError error;
	DBusConnection *conn = connect_to_system_bus();

	const char *manager_path = "/org/freedesktop/systemd1";
	const char *manager_interface = "org.freedesktop.systemd1.Manager";
	const char *method = "StopUnit";

	DBusMessage *msg = dbus_message_new_method_call(
		"org.freedesktop.systemd1",
		manager_path,
		manager_interface,
		method
	);

	const char *mode = "replace"; // Stop and replace existing job
	dbus_message_append_args(msg, DBUS_TYPE_STRING, &service_name,
			DBUS_TYPE_STRING, &mode, DBUS_TYPE_INVALID);

	DBusMessage *reply =
			dbus_connection_send_with_reply_and_block(conn,
					msg, 5000, &error);

	dbus_message_unref(reply);
	dbus_message_unref(msg);
	dbus_connection_unref(conn);

	return 0;
}

with the previously mentioned problem that now xfsprogs grows build and packaging dependencies on libdbus. My guess is that it'll be a long time till they deprecate starting services over dbus. AFAICT systemd in Trixie doesn't even expose varlink endpoints yet.

(Note that xfs_scrub_all already has a runtime dependency on python3-dbus)

--D

#9 lore raw

Re: [PATCH 03/26] libfrog: add support code for starting systemd services programmatically

Quoted 71 lines · Darrick J. Wong
Quoted 16 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some simple routines for computing the name of systemd service instances and starting systemd services. These will be used by the xfs_healer_start service to start per-filesystem xfs_healer service instances.

Quoted 7 lines

Note that we run systemd helper programs as subprocesses for a couple of reasons. First, the path-escaping functionality is not a part of any library-accessible API, which means it can only be accessed via systemd-escape(1). Second, although the service startup functionality can be reached via dbus, doing so would introduce a new library dependency. Systemd is also undergoing a dbus -> varlink RPC transition so we avoid that mess by calling the cli systemctl(1) program.

Quoted 4 lines

Just curious: did you run this past the systemd folks? Shelling out always feel a bit iffy, and they're usually happy to help on how to integrate with their services, so just asking might result in a better way.

I'll do that, though even if they add a dbus/varlink endpoint for path escaping, it'll be a few releases more until we can depend on it existing in the distros. :(

Service startup can be done fairly easily with something like this, though obviously you'd want to do real error checking here:

static DBusConnection*
connect_to_system_bus(void)

{
	// Connect to the system bus (requires root or polkit permissions)
	DBusConnection *conn = dbus_bus_get(DBUS_BUS_SYSTEM, error);

	return conn;
}

int
systemd_stop_service(
const char *service_name)

{
	DBusError error;
	DBusConnection *conn = connect_to_system_bus();

	const char *manager_path = "/org/freedesktop/systemd1";
	const char *manager_interface = "org.freedesktop.systemd1.Manager";
	const char *method = "StopUnit";

	DBusMessage *msg = dbus_message_new_method_call(
		"org.freedesktop.systemd1",
		manager_path,
		manager_interface,
		method
	);

	const char *mode = "replace"; // Stop and replace existing job
	dbus_message_append_args(msg, DBUS_TYPE_STRING, &service_name,
			DBUS_TYPE_STRING, &mode, DBUS_TYPE_INVALID);

	DBusMessage *reply =
			dbus_connection_send_with_reply_and_block(conn,
					msg, 5000, &error);

	dbus_message_unref(reply);
	dbus_message_unref(msg);
	dbus_connection_unref(conn);

	return 0;
}
Quoted 7 lines

with the previously mentioned problem that now xfsprogs grows build and packaging dependencies on libdbus. My guess is that it'll be a long time till they deprecate starting services over dbus. AFAICT systemd in Trixie doesn't even expose varlink endpoints yet.

(Note that xfs_scrub_all already has a runtime dependency on python3-dbus)

Interim update -- just from looking at what
'systemctl restart --no-block' does, there's quite a lot of complexity that the CLI hides. If you ask it to start a service, it gives you back a job object, then you can sit and wait to see what the result of the job is, etc. The above vibecoding was actually enough to start the service, but TBH I think the dangers of shelling out are <cough> roughly on the same level as all the crap you have to add to talk to systemd over libdbus.

I'll try their mailing list, but first I have to wait for them to approve my subscription, and only then can I ask...

--D

#10 lore raw

Re: [PATCH 03/26] libfrog: add support code for starting systemd services programmatically

Quoted 11 lines · Darrick J. Wong

Interim update -- just from looking at what
'systemctl restart --no-block' does, there's quite a lot of complexity that the CLI hides. If you ask it to start a service, it gives you back a job object, then you can sit and wait to see what the result of the job is, etc. The above vibecoding was actually enough to start the service, but TBH I think the dangers of shelling out are <cough> roughly on the same level as all the crap you have to add to talk to systemd over libdbus.

I'll try their mailing list, but first I have to wait for them to approve my subscription, and only then can I ask...

Eh. But yeah, I guess just sticking to the CLI might end up best.

#11 lore raw

PATCH 4/26 libfrog: hoist a couple of service helper functions

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Hoist a couple of service/daemon-related helper functions to libfrog so that we can share the code between xfs_scrub and xfs_healer.

---
 libfrog/systemd.h |   28 ++++++++++++++++++++++++++++
 scrub/xfs_scrub.c |   32 +++++++++-----------------------
 2 files changed, 37 insertions(+), 23 deletions(-)
libfrog/systemd.h +28 -0
index 4f414bc3c1e9c3..c96df4afa39aa6 100644
@@ -17,4 +17,32 @@ enum systemd_unit_manage {
 
 int systemd_manage_unit(enum systemd_unit_manage how, const char *unitname);
 
+static inline bool systemd_is_service(void)
+{
+	return getenv("SERVICE_MODE") != NULL;
+}
+
+/* Special processing for a service/daemon program that is exiting. */
+static inline int
+systemd_service_exit(int ret)
+{
+	/*
+	 * We have to sleep 2 seconds here because journald uses the pid to
+	 * connect our log messages to the systemd service.  This is critical
+	 * for capturing all the log messages if the service fails, because
+	 * failure analysis tools use the service name to gather log messages
+	 * for reporting.
+	 */
+	sleep(2);
+
+	/*
+	 * If we're being run as a service, the return code must fit the LSB
+	 * init script action error guidelines, which is to say that we
+	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
+	 * section 22.2) and hope the admin will scan the log for what actually
+	 * happened.
+	 */
+	return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+}
+
 #endif /* __LIBFROG_SYSTEMD_H__ */
scrub/xfs_scrub.c +9 -23
index 3dba972a7e8d2a..79937aa8cce4c4 100644
@@ -19,6 +19,7 @@
 #include "unicrash.h"
 #include "progress.h"
 #include "libfrog/histogram.h"
+#include "libfrog/systemd.h"
 
 /*
  * XFS Online Metadata Scrub (and Repair)
@@ -866,8 +867,7 @@ main(
 	if (stdout_isatty && !progress_fp)
 		progress_fp = fdopen(1, "w+");
 
-	if (getenv("SERVICE_MODE"))
-		is_service = true;
+	is_service = systemd_is_service();
 
 	/* Initialize overall phase stats. */
 	error = phase_start(&all_pi, 0, NULL);
@@ -960,29 +960,15 @@ main(
 	hist_free(&ctx.datadev_hist);
 	hist_free(&ctx.rtdev_hist);
 
-	/*
-	 * If we're being run as a service, the return code must fit the LSB
-	 * init script action error guidelines, which is to say that we
-	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
-	 * section 22.2) and hope the admin will scan the log for what
-	 * actually happened.
-	 *
-	 * We have to sleep 2 seconds here because journald uses the pid to
-	 * connect our log messages to the systemd service.  This is critical
-	 * for capturing all the log messages if the scrub fails, because the
-	 * fail service uses the service name to gather log messages for the
-	 * error report.
-	 *
-	 * Note: We don't count a lack of kernel support as a service failure
-	 * because we haven't determined that there's anything wrong with the
-	 * filesystem.
-	 */
 	if (is_service) {
-		sleep(2);
+		/*
+		 * Note: We don't count a lack of kernel support as a service
+		 * failure because we haven't determined that there's anything
+		 * wrong with the filesystem.
+		 */
 		if (!ctx.scrub_setup_succeeded)
-			return 0;
-		if (ret != SCRUB_RET_SUCCESS)
-			return 1;
+			ret = 0;
+		return systemd_service_exit(ret);
 	}
 
 	return ret;
#13 lore raw

PATCH 5/26 man2: document the healthmon ioctl

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Document the XFS_IOC_HEALTH_MONITOR and
XFS_IOC_HEALTH_FD_ON_MONITORED_FS ioctls.

---
 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 |   75 ++++
 man/man2/ioctl_xfs_health_monitor.2            |  464 ++++++++++++++++++++++++
 2 files changed, 539 insertions(+)
 create mode 100644 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2
 create mode 100644 man/man2/ioctl_xfs_health_monitor.2
man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 new file +75 -0
new file mode 100644
index 00000000000000..bbc5ce9bbabf53
@@ -0,0 +1,75 @@
+.\" Copyright (c) 2025-2026, Oracle.  All rights reserved.
+.\"
+.\" %%%LICENSE_START(GPLv2+_DOC_FULL)
+.\" SPDX-License-Identifier: GPL-2.0+
+.\" %%%LICENSE_END
+.TH IOCTL-XFS-HEALTH-FD-ON-MONITORED-FS 2 2026-01-04 "XFS"
+.SH NAME
+ioctl_xfs_health_fd_on_monitored_fs \- check if the given fd belongs to the same fs being monitored
+.SH SYNOPSIS
+.br
+.B #include <xfs/xfs_fs.h>
+.PP
+.BI "int ioctl(int " healthmon_fd ", XFS_IOC_HEALTH_FD_ON_MONITORED_FS, struct xfs_health_file_on_monitored_fs *" arg );
+.SH DESCRIPTION
+This XFS healthmon fd ioctl asks the kernel driver if the file descriptor
+passed in via
+.I arg
+points to a file on the same filesystem that is being monitored by
+.IR healthmon_fd .
+The file descriptor is conveyed in a structure of the following form:
+.PP
+.in +4n
+.nf
+struct xfs_health_file_on_monitored_fs {
+	__s32 fd;
+	__u32 flags;
+};
+.fi
+.in
+.PP
+The field
+.I flags
+must be zero.
+.PP
+The field
+.I fd
+is a descriptor of an open file.
+.PP
+The argument
+.I healthmon_fd
+must be a file opened via the
+.B XFS_IOC_HEALTH_MONITOR
+ioctl.
+.SH RETURN VALUE
+On error, \-1 is returned, and
+.I errno
+is set to indicate the error.
+If the file descriptor points to a file on the same filesystem that is being
+monitored, 0 is returned.
+.PP
+.SH ERRORS
+Error codes can be one of, but are not limited to, the following:
+.TP
+.B ESTALE
+The open file is not on the same filesystem that is being monitored.
+.TP
+.B EINVAL
+One or more of the arguments specified is invalid.
+.TP
+.B EBADF
+.I arg.fd
+does not refer to an open file.
+.TP
+.B EFAULT
+The
+.I arg
+structure could not be copied into the kernel.
+.TP
+.B ENOTTY
+.I healthmon_fd
+is not a XFS health monitoring file.
+.SH CONFORMING TO
+This API is specific to XFS filesystem on the Linux kernel.
+.SH SEE ALSO
+.BR ioctl_xfs_health_monitor (2)
man/man2/ioctl_xfs_health_monitor.2 new file +464 -0

464 lines not on this page · raw message

#15 lore raw

PATCH 6/26 man2: document the media verification ioctl

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Document XFS_IOC_VERIFY_MEDIA, which is a new ioctl for xfs_scrub to perform media scans on the disks underneath the filesystem. This will enable media errors to be reported to xfs_healer and fsnotify.

---
 man/man2/ioctl_xfs_verify_media.2 |  185 +++++++++++++++++++++++++++++++++++++
 1 file changed, 185 insertions(+)
 create mode 100644 man/man2/ioctl_xfs_verify_media.2
man/man2/ioctl_xfs_verify_media.2 new file +185 -0
new file mode 100644
index 00000000000000..bd0d4579f5a364
@@ -0,0 +1,185 @@
+.\" Copyright (c) 2025-2026, Oracle.  All rights reserved.
+.\"
+.\" %%%LICENSE_START(GPLv2+_DOC_FULL)
+.\" SPDX-License-Identifier: GPL-2.0+
+.\" %%%LICENSE_END
+.TH IOCTL-XFS-VERIFY-MEDIA 2 2026-01-09 "XFS"
+.SH NAME
+ioctl_xfs_verify_media \- verify the media of the devices backing XFS
+.SH SYNOPSIS
+.br
+.B #include <xfs/xfs_fs.h>
+.PP
+.BI "int ioctl(int " fd ", XFS_IOC_VERIFY_MEDIA, struct xfs_verify_media *" arg );
+.SH DESCRIPTION
+Verify the media of a storage device backing an XFS filesystem.
+If errors are found, report the error to the kernel so that it can generate
+health events for the health monitoring system and fsnotify.
+The verification request is conveyed in a structure of the following form:
+.PP
+.in +4n
+.nf
+struct xfs_verify_error {
+	__u32	me_dev;
+	__u32	me_flags;
+	__u64	me_start_daddr;
+	__u64	me_end_daddr;
+	__u32	me_ioerror;
+	__u32	me_pad;
+};
+.fi
+.in
+.PP
+The field
+.I me_pad
+must be zero.
+.PP
+The field
+.I me_ioerror
+will be set if the ioctl returns success.
+.PP
+The fields
+.I me_start_daddr
+and
+.I me_end_daddr
+are the range of the storage device to verify.
+Both values must be in units of 512-byte blocks.
+The
+.I me_start_daddr
+field is inclusive, and the
+.I me_end_daddr
+field is exclusive.
+If
+.I me_end_daddr
+is larger than the size of the device, the kernel will set it to the size of
+the device.
+
+If the system call returns success and any part of the storage device range was
+successfully verified, the
+.I me_start_daddr
+field will be updated to reflect the successful verification.
+If after this update the
+.I me_start_daddr
+is equal to
+.IR me_end_daddr ,
+then the entire range was verified successfully.
+
+If not, then a media error was encountered and the caller should generate a
+series of secondary calls to this ioctl with smaller ranges to discover the
+exact location and type of media error.
+The type of media error will be written to the
+.I me_ioerror
+field.
+
+.PP
+The field
+.I me_dev
+must be one of the following values:
+.RS 0.4i
+.TP
+.B XFS_DEV_DATA
+Verify the data device.
+.TP
+.B XFS_DEV_LOG
+Verify the external log device.
+.TP
+.B XFS_DEV_RT
+Verify the realtime device.
+.RE
+.PP
+The field
+.I me_flags
+is a bitmask of one of the following values:
+.RS 0.4i
+.TP
+.B XFS_VERIFY_MEDIA_REPORT
+Report all media errors to fsnotify.
+.RE
+
+The
+.IR me_max_io_size
+field, if nonzero, will be used as advice for the maximum size of the IO to
+send to the device.
+
+The
+.I me_rest_us
+field will cause the kernel to pause for this many microseconds between IO
+requests.
+
+.SH RETURN VALUE
+On runtime error, \-1 is returned, and
+.I errno
+is set to indicate the error.
+If 0 is returned, then
+.I start_daddr
+or
+.I ioerror
+will be updated.
+.PP
+.SH ERRORS
+Error codes can be one of, but are not limited to, the following:
+.TP
+.B EPERM
+The calling process does not have sufficient privilege.
+.TP
+.B EINVAL
+One or more of the arguments specified is invalid.
+.TP
+.B EFAULT
+The
+.I arg
+structure could not be copied into the kernel.
+.TP
+.B ENODEV
+The device is not present.
+.TP
+.B ENOMEM
+There was not enough memory to perform the verification.
+
+.SH I/O ERRORS
+The
+.I ioerror
+field could be set to one of the following:
+.TP
+.B 0
+The verification I/O succeeded.
+.TP
+.B EOPNOTSUPP
+.TP
+.B ETIMEDOUT
+The kernel timed out the verification I/O command.
+.TP
+.B ENOLINK
+The transportation link to the storage device was down temporarily.
+.TP
+.B EREMOTEIO
+The storage target controller suffered a critical error.
+.TP
+.B ENODATA
+The storage target media suffered a critical error.
+.TP
+.B EILSEQ
+Storage protection metadata did not validate successfully.
+.TP
+.B ENOMEM
+There was not enough memory to allocate an I/O request.
+.TP
+.B ENODEV
+The storage device is offline.
+.TP
+.B ETIME
+The storage device timed out the I/O command.
+.TP
+.B EINVAL
+The I/O request was rejected by the device for being invalid.
+.TP
+.B EIO
+An I/O error occurred but no specific details are available.
+.RE
+.PP
+This list is not exhaustive and may grow in the future.
+
+.SH CONFORMING TO
+This API is specific to XFS filesystem on the Linux kernel.
+.SH SEE ALSO
+.BR ioctl_xfs_health_monitor (2)
#17 lore raw

PATCH 7/26 xfs_io: monitor filesystem health events

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a subcommand to monitor for health events generated by the kernel.

---
 io/io.h           |    1 
 io/Makefile       |    1 
 io/healthmon.c    |  186 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 io/init.c         |    1 
 man/man8/xfs_io.8 |   25 +++++++
 5 files changed, 214 insertions(+)
 create mode 100644 io/healthmon.c
io/io.h +1 -0
index 35fb8339eeb5aa..2f5262bce6acbb 100644
@@ -162,3 +162,4 @@ extern void		bulkstat_init(void);
 void			exchangerange_init(void);
 void			fsprops_init(void);
 void			aginfo_init(void);
+void			healthmon_init(void);
io/Makefile +1 -0
index 444e2d6a557d5d..8e3783353a52b5 100644
@@ -25,6 +25,7 @@ CFILES = \
 	fsuuid.c \
 	fsync.c \
 	getrusage.c \
+	healthmon.c \
 	imap.c \
 	init.c \
 	inject.c \
io/healthmon.c new file +186 -0

186 lines not on this page · raw message

io/init.c +1 -0
index 49e9e7cb88214b..cb5573f45ccfbc 100644
@@ -92,6 +92,7 @@ init_commands(void)
 	crc32cselftest_init();
 	exchangerange_init();
 	fsprops_init();
+	healthmon_init();
 }
 
 /*
man/man8/xfs_io.8 +25 -0
index 0a673322fde3a1..f7f2956a54a7aa 100644
@@ -1356,6 +1356,31 @@ .SH FILESYSTEM COMMANDS
 .B thaw
 Undo the effects of a filesystem freeze operation.
 Only available in expert mode and requires privileges.
+.TP
+.BI "healthmon [ \-c " bufsize " ] [ \-c ] [ \-d " delay_ms " ] [ \-p ] [ \-v ]"
+Watch for filesystem health events and write them to the console.
+.RE
+.RS 1.0i
+.PD 0
+.TP
+.BI "\-b " bufsize
+Use a buffer of this size to read events from the kernel.
+.TP
+.BI \-c
+Close the open file and replace it with the monitor file.
+.TP
+.BI "\-d " delay_ms
+Sleep for this long between read attempts.
+.TP
+.B \-p
+Probe for the existence of the functionality by opening the monitoring fd and
+closing it immediately.
+.TP
+.BI \-v
+Request all health events, even if nothing changed.
+.PD
+.RE
+
 .TP
 .BI "inject [ " tag " ]"
 Inject errors into a filesystem to observe filesystem behavior at
#19 lore raw

PATCH 8/26 xfs_io: add a media verify command

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a subcommand to invoke the media verification ioctl to make sure that we can actually check the storage underneath an xfs filesystem.

---
 io/io.h           |    1 
 io/Makefile       |    3 +
 io/init.c         |    1 
 io/verify_media.c |  180 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_io.8 |   42 ++++++++++++
 5 files changed, 226 insertions(+), 1 deletion(-)
 create mode 100644 io/verify_media.c
io/io.h +1 -0
index 2f5262bce6acbb..0f12b3cfed5e76 100644
@@ -163,3 +163,4 @@ void			exchangerange_init(void);
 void			fsprops_init(void);
 void			aginfo_init(void);
 void			healthmon_init(void);
+void			verifymedia_init(void);
io/Makefile +2 -1
index 8e3783353a52b5..79d5e172b8f31f 100644
@@ -51,7 +51,8 @@ CFILES = \
 	sync.c \
 	sync_file_range.c \
 	truncate.c \
-	utimes.c
+	utimes.c \
+	verify_media.c
 
 LLDLIBS = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG) $(LIBPTHREAD) $(LIBUUID)
 LTDEPENDENCIES = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG)
io/init.c +1 -0
index cb5573f45ccfbc..f2a551ef559200 100644
@@ -93,6 +93,7 @@ init_commands(void)
 	exchangerange_init();
 	fsprops_init();
 	healthmon_init();
+	verifymedia_init();
 }
 
 /*
io/verify_media.c new file +180 -0

180 lines not on this page · raw message

man/man8/xfs_io.8 +42 -0

48 lines not on this page · raw message

#21 lore raw

PATCH 9/26 xfs_healer: create daemon to listen for health events

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a daemon program that can listen for and log health events. Eventually this will be used to self-heal filesystems in real time.

Because events can take a while to process, the main thread reads event objects from the healthmon fd and dispatches them to a background workqueue as quickly as it can. This split of responsibilities is necessary because the kernel event queue will drop events if the queue fills up, and each event can take some time to process (logging, repairs, etc.) so we don't want to lose events.

To be clear, xfs_healer and xfs_scrub are complementary tools:

Scrub walks the whole filesystem, finds stuff that needs fixing or rebuilding, and rebuilds it. This is sort of analogous to a patrol scrub.

Healer listens for metadata corruption messages from the kernel and issues a targeted repair of that structure. This is kind of like an ondemand scrub.

My end goal is that xfs_healer (the service) is active all the time and can respond instantly to a corruption report, whereas xfs_scrub (the service) gets run periodically as a cron job.

xfs_healer can decide that it's overwhelmed with problems and start xfs_scrub to deal with the mess. Ideally you don't crash the filesystem and then have to use xfs_repair to smash your way back to a mountable filesystem.

By default we run xfs_healer as a background service, which means that we only start two threads -- one to read the events, and another to process them. In other words, we try not to use all available hardware resources for repairs. The foreground mode switch starts up a large number of threads to try to increase parallelism, which may or may not be useful for repairs depending on how much metadata the kernel needs to scan.

---
 healer/xfs_healer.h  |   47 ++++++
 Makefile             |    5 +
 configure.ac         |    6 +
 healer/Makefile      |   35 +++++
 healer/xfs_healer.c  |  377 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/builddefs.in |    1 
 6 files changed, 471 insertions(+)
 create mode 100644 healer/xfs_healer.h
 create mode 100644 healer/Makefile
 create mode 100644 healer/xfs_healer.c
healer/xfs_healer.h new file +47 -0
new file mode 100644
index 00000000000000..bcddde5db0cc47
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025-2026 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef XFS_HEALER_XFS_HEALER_H_
+#define XFS_HEALER_XFS_HEALER_H_
+
+extern char *progname;
+
+/*
+ * When running in environments with restrictive security policies, healer
+ * might not be allowed to access the global mount tree.  However, processes
+ * are usually still allowed to see their own mount tree, so use this path for
+ * all mount table queries.
+ */
+#define _PATH_PROC_MOUNTS	"/proc/self/mounts"
+
+struct healer_ctx {
+	/* CLI options, must be int */
+	int			debug;
+	int			log;
+	int			everything;
+	int			foreground;
+
+	/* fd and fs geometry for mount */
+	struct xfs_fd		mnt;
+
+	/* Shared reference to the user's mountpoint for logging */
+	const char		*mntpoint;
+
+	/* Shared reference to the getmntent fsname for reconnecting */
+	const char		*fsname;
+
+	/* file stream of monitor and buffer */
+	FILE			*mon_fp;
+	char			*mon_buf;
+
+	/* coordinates logging printfs */
+	pthread_mutex_t		conlock;
+
+	/* event queue */
+	struct workqueue	event_queue;
+	bool			queue_active;
+};
+
+#endif /* XFS_HEALER_XFS_HEALER_H_ */
Makefile +5 -0
index c73aa391bc5f43..1f499c30f3457e 100644
@@ -69,6 +69,10 @@ ifeq ("$(ENABLE_SCRUB)","yes")
 TOOL_SUBDIRS += scrub
 endif
 
+ifeq ("$(ENABLE_HEALER)","yes")
+TOOL_SUBDIRS += healer
+endif
+
 ifneq ("$(XGETTEXT)","")
 TOOL_SUBDIRS += po
 endif
@@ -100,6 +104,7 @@ mkfs: libxcmd
 spaceman: libxcmd libhandle
 scrub: libhandle libxcmd
 rtcp: libfrog
+healer: libhandle
 
 ifeq ($(HAVE_BUILDDEFS), yes)
 include $(BUILDRULES)
configure.ac +6 -0
index 8d2bbb9ef88bb9..78bb87b159b10b 100644
@@ -110,6 +110,12 @@ AC_ARG_ENABLE(libicu,
 [  --enable-libicu=[yes/no]  Enable Unicode name scanning in xfs_scrub (libicu) [default=probe]],,
 	enable_libicu=probe)
 
+# Enable xfs_healer build
+AC_ARG_ENABLE(healer,
+[  --enable-healer=[yes/no]  Enable build of xfs_healer utility [[default=yes]]],,
+	enable_healer=yes)
+AC_SUBST(enable_healer)
+
 #
 # If the user specified a libdir ending in lib64 do not append another
 # 64 to the library names.
healer/Makefile new file +35 -0
new file mode 100644
index 00000000000000..e82c820883669a
@@ -0,0 +1,35 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024-2026 Oracle.  All Rights Reserved.
+#
+
+TOPDIR = ..
+builddefs=$(TOPDIR)/include/builddefs
+include $(builddefs)
+
+INSTALL_HEALER = install-healer
+
+LTCOMMAND = xfs_healer
+
+CFILES = \
+xfs_healer.c
+
+HFILES = \
+xfs_healer.h
+
+LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD)
+LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG)
+LLDFLAGS = -static
+
+default: depend $(LTCOMMAND)
+
+include $(BUILDRULES)
+
+install: $(INSTALL_HEALER)
+
+install-healer: default
+	$(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR)
+	$(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR)
+
+install-dev:
+
+-include .dep
healer/xfs_healer.c new file +377 -0

377 lines not on this page · raw message

include/builddefs.in +1 -0
index 4a2cb757c0bdb3..99373ec86215cf 100644
@@ -91,6 +91,7 @@ ENABLE_SHARED	= @enable_shared@
 ENABLE_GETTEXT	= @enable_gettext@
 ENABLE_EDITLINE	= @enable_editline@
 ENABLE_SCRUB	= @enable_scrub@
+ENABLE_HEALER	= @enable_healer@
 
 HAVE_ZIPPED_MANPAGES = @have_zipped_manpages@
 
#23 lore raw

PATCH 10/26 xfs_healer: enable repairing filesystems

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make it so that our health monitoring daemon can initiate repairs in response to reports of corrupt filesystem metadata. Repairs are initiated from the background workers as explained in the previous patch.

Note that just like xfs_scrub, xfs_healer's ability to repair metadata relies heavily on back references such as reverse mappings and directory parent pointers to add redundancy to the filesystem. Check for these two features and whine a bit if they are missing, just like scrub.

---
 healer/xfs_healer.h   |   28 ++++++
 libfrog/flagmap.h     |    3 +
 libfrog/healthevent.h |   12 ++
 healer/Makefile       |    2 
 healer/fsrepair.c     |  249 +++++++++++++++++++++++++++++++++++++++++++++++++
 healer/weakhandle.c   |  115 +++++++++++++++++++++++
 healer/xfs_healer.c   |   56 +++++++++++
 libfrog/flagmap.c     |   17 +++
 libfrog/healthevent.c |  117 +++++++++++++++++++++++
 9 files changed, 599 insertions(+)
 create mode 100644 healer/fsrepair.c
 create mode 100644 healer/weakhandle.c
healer/xfs_healer.h +28 -0
index bcddde5db0cc47..a4de1ad32a408f 100644
@@ -8,6 +8,9 @@
 
 extern char *progname;
 
+struct weakhandle;
+struct hme_prefix;
+
 /*
  * When running in environments with restrictive security policies, healer
  * might not be allowed to access the global mount tree.  However, processes
@@ -22,6 +25,7 @@ struct healer_ctx {
 	int			log;
 	int			everything;
 	int			foreground;
+	int			want_repair;
 
 	/* fd and fs geometry for mount */
 	struct xfs_fd		mnt;
@@ -32,6 +36,9 @@ struct healer_ctx {
 	/* Shared reference to the getmntent fsname for reconnecting */
 	const char		*fsname;
 
+	/* weak file handle so we can reattach to filesystem */
+	struct weakhandle	*wh;
+
 	/* file stream of monitor and buffer */
 	FILE			*mon_fp;
 	char			*mon_buf;
@@ -44,4 +51,25 @@ struct healer_ctx {
 	bool			queue_active;
 };
 
+static inline bool healer_has_rmapbt(const struct healer_ctx *ctx)
+{
+	return ctx->mnt.fsgeom.flags & XFS_FSOP_GEOM_FLAGS_RMAPBT;
+}
+
+static inline bool healer_has_parent(const struct healer_ctx *ctx)
+{
+	return ctx->mnt.fsgeom.flags & XFS_FSOP_GEOM_FLAGS_PARENT;
+}
+
+/* repair.c */
+int repair_metadata(struct healer_ctx *ctx, const struct hme_prefix *pfx,
+		const struct xfs_health_monitor_event *hme);
+bool healer_can_repair(struct healer_ctx *ctx);
+
+/* weakhandle.c */
+int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
+		struct weakhandle **whp);
+int weakhandle_reopen(struct weakhandle *wh, int *fd);
+void weakhandle_free(struct weakhandle **whp);
+
 #endif /* XFS_HEALER_XFS_HEALER_H_ */
libfrog/flagmap.h +3 -0
index 8031d75a7c02a8..05110c3544dc97 100644
@@ -14,6 +14,9 @@ struct flag_map {
 void mask_to_string(const struct flag_map *map, unsigned long long mask,
 		const char *delimiter, char *buf, size_t bufsize);
 
+const char *lowest_set_mask_string(const struct flag_map *map,
+		unsigned long long mask);
+
 const char *value_to_string(const struct flag_map *map,
 		unsigned long long value);
 
libfrog/healthevent.h +12 -0
index 6de41bc797100c..4f3c8ba639ec4c 100644
@@ -40,4 +40,16 @@ hme_prefix_init(
 void hme_report_event(const struct hme_prefix *pfx,
 		const struct xfs_health_monitor_event *hme);
 
+enum repair_outcome {
+	REPAIR_SUCCESS,
+	REPAIR_FAILED,
+	REPAIR_PROBABLY_OK,
+	REPAIR_UNNECESSARY,
+};
+
+void report_health_repair(const struct hme_prefix *pfx,
+		const struct xfs_health_monitor_event *hme,
+		uint32_t event_mask,
+		enum repair_outcome outcome);
+
 #endif /* LIBFROG_HEALTHEVENT_H_ */
healer/Makefile +2 -0
index e82c820883669a..981192b81af626 100644
@@ -11,6 +11,8 @@ INSTALL_HEALER = install-healer
 LTCOMMAND = xfs_healer
 
 CFILES = \
+fsrepair.c \
+weakhandle.c \
 xfs_healer.c
 
 HFILES = \
healer/fsrepair.c new file +249 -0

249 lines not on this page · raw message

healer/weakhandle.c new file +115 -0

115 lines not on this page · raw message

healer/xfs_healer.c +56 -0

104 lines not on this page · raw message

libfrog/flagmap.c +17 -0
index 631c4bbc8f1dc0..ce413297780a2a 100644
@@ -44,6 +44,23 @@ mask_to_string(
 		snprintf(buf, bufsize, "%s0x%llx", tag, mask & ~seen);
 }
 
+/*
+ * Given a mapping of bits to strings and a bitmask, return the string
+ * corresponding to the lowest set bit in the mask.
+ */
+const char *
+lowest_set_mask_string(
+	const struct flag_map	*map,
+	unsigned long long	mask)
+{
+	for (; map->string; map++) {
+		if (mask & map->flag)
+			return _(map->string);
+	}
+
+	return _("unknown flag");
+}
+
 /*
  * Given a mapping of values to strings and a value, return the matching string
  * or confusion.
libfrog/healthevent.c +117 -0

120 lines not on this page · raw message

#25 lore raw

PATCH 11/26 xfs_healer: use getparents to look up file names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the kernel tells about something that happened to a file, use the GETPARENTS ioctl to try to look up the path to that file for more ergonomic reporting.

---
 healer/xfs_healer.h |    6 ++++
 healer/fsrepair.c   |   16 ++++++++-
 healer/weakhandle.c |   86 +++++++++++++++++++++++++++++++++++++++++++++++++++
 healer/xfs_healer.c |   45 ++++++++++++++++++++++++++-
 4 files changed, 149 insertions(+), 4 deletions(-)
healer/xfs_healer.h +6 -0
index a4de1ad32a408f..6d12921245934c 100644
@@ -61,6 +61,10 @@ static inline bool healer_has_parent(const struct healer_ctx *ctx)
 	return ctx->mnt.fsgeom.flags & XFS_FSOP_GEOM_FLAGS_PARENT;
 }
 
+void lookup_path(struct healer_ctx *ctx,
+		const struct xfs_health_monitor_event *hme,
+		struct hme_prefix *pfx);
+
 /* repair.c */
 int repair_metadata(struct healer_ctx *ctx, const struct hme_prefix *pfx,
 		const struct xfs_health_monitor_event *hme);
@@ -71,5 +75,7 @@ int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
 		struct weakhandle **whp);
 int weakhandle_reopen(struct weakhandle *wh, int *fd);
 void weakhandle_free(struct weakhandle **whp);
+int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
+		char *path, size_t pathlen);
 
 #endif /* XFS_HEALER_XFS_HEALER_H_ */
healer/fsrepair.c +14 -2
index 907afca3dba8a7..4534104f8a6ac1 100644
@@ -164,7 +164,7 @@ try_repair_rtgroup(
 static void
 try_repair_inode(
 	struct healer_ctx			*ctx,
-	const struct hme_prefix			*pfx,
+	const struct hme_prefix			*orig_pfx,
 	int					mnt_fd,
 	const struct xfs_health_monitor_event	*hme)
 {
@@ -182,13 +182,25 @@ try_repair_inode(
 		{0,		0},
 	};
 #undef X
-	const struct u32_scrub *f;
+	struct hme_prefix	new_pfx;
+	const struct hme_prefix	*pfx = orig_pfx;
+	const struct u32_scrub	*f;
 
 	foreach_scrub_type(f, hme->e.inode.mask, INODE_STRUCTURES) {
 		enum repair_outcome	outcome =
 			xfs_repair_metadata(mnt_fd, f->scrub_type,
 					0, hme->e.inode.ino, hme->e.inode.gen);
 
+		/*
+		 * Try again to find the file path, maybe we fixed the dir
+		 * tree.
+		 */
+		if (!hme_prefix_has_path(pfx)) {
+			lookup_path(ctx, hme, &new_pfx);
+			if (hme_prefix_has_path(&new_pfx))
+				pfx = &new_pfx;
+		}
+
 		pthread_mutex_lock(&ctx->conlock);
 		report_health_repair(pfx, hme, f->event_mask, outcome);
 		pthread_mutex_unlock(&ctx->conlock);
healer/weakhandle.c +86 -0

95 lines not on this page · raw message

healer/xfs_healer.c +43 -2

64 lines not on this page · raw message

#27 lore raw

PATCH 12/26 xfs_healer: create a per-mount background monitoring service

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a systemd service definition for our self-healing filesystem daemon so that we can run it for every mounted filesystem.

---
 healer/Makefile                |   19 +++++++
 healer/system-xfs_healer.slice |   31 ++++++++++++
 healer/xfs_healer.c            |    3 +
 healer/[email protected]  |  107 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 158 insertions(+), 2 deletions(-)
 create mode 100644 healer/system-xfs_healer.slice
 create mode 100644 healer/[email protected]
healer/Makefile +18 -1
index 981192b81af626..86d6f50781f9b6 100644
@@ -22,7 +22,20 @@ LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD)
 LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG)
 LLDFLAGS = -static
 
-default: depend $(LTCOMMAND)
+ifeq ($(HAVE_SYSTEMD),yes)
+INSTALL_HEALER += install-systemd
+SYSTEMD_SERVICES=\
+	system-xfs_healer.slice \
+	[email protected]
+OPTIONAL_TARGETS += $(SYSTEMD_SERVICES)
+endif
+
+default: depend $(LTCOMMAND) $(SYSTEMD_SERVICES)
+
+%.service: %.service.in $(builddefs)
+	@echo "    [SED]    $@"
+	$(Q)$(SED) -e "s|@pkg_libexec_dir@|$(PKG_LIBEXEC_DIR)|g" \
+		   < $< > $@
 
 include $(BUILDRULES)
 
@@ -32,6 +45,10 @@ install-healer: default
 	$(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR)
 	$(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR)
 
+install-systemd: default
+	$(INSTALL) -m 755 -d $(SYSTEMD_SYSTEM_UNIT_DIR)
+	$(INSTALL) -m 644 $(SYSTEMD_SERVICES) $(SYSTEMD_SYSTEM_UNIT_DIR)
+
 install-dev:
 
 -include .dep
healer/system-xfs_healer.slice new file +31 -0
new file mode 100644
index 00000000000000..b8f5bca03963ff
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+# Author: Darrick J. Wong <[email protected]>
+
+[Unit]
+Description=xfs_healer background service slice
+Before=slices.target
+
+[Slice]
+
+# If the CPU usage cgroup controller is available, don't use more than 2 cores
+# for all background processes.  One thread to read events, another to run
+# repairs.
+CPUQuota=200%
+CPUAccounting=true
+
+[Install]
+# As of systemd 249, the systemd cgroupv2 configuration code will drop resource
+# controllers from the root and system.slice cgroups at startup if it doesn't
+# find any direct dependencies that require a given controller.  Newly
+# activated units with resource control directives are created under the system
+# slice but do not cause a reconfiguration of the slice's resource controllers.
+# Hence we cannot put CPUQuota= into the xfs_healer service units directly.
+#
+# For the CPUQuota directive to have any effect, we must therefore create an
+# explicit definition file for the slice that systemd creates to contain the
+# xfs_healer instance units (e.g. [email protected]) and we must configure
+# this slice as a dependency of the system slice to establish the direct
+# dependency relation.
+WantedBy=system.slice
healer/xfs_healer.c +2 -1
index c9892168c706cb..2f431f18c69318 100644
@@ -12,6 +12,7 @@
 #include "libfrog/paths.h"
 #include "libfrog/healthevent.h"
 #include "libfrog/workqueue.h"
+#include "libfrog/systemd.h"
 #include "xfs_healer.h"
 
 /* Program name; needed for libfrog error reports. */
@@ -470,5 +471,5 @@ main(
 	teardown_monitor(&ctx);
 	free((char *)ctx.fsname);
 out:
-	return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+	return systemd_service_exit(ret);
 }
healer/[email protected] new file +107 -0

107 lines not on this page · raw message

#29 lore raw

PATCH 13/26 xfs_healer: create a service to start the per-mount healer service

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a daemon to wait for xfs mount events via fsnotify and start up the per-mount healer service. It's important that we're running in the same mount namespace as the mount, so we're a fanotify client to avoid having to filter the mount namespaces ourselves.

---
 libfrog/systemd.h                  |   23 +-
 configure.ac                       |    5 
 healer/Makefile                    |   27 ++-
 healer/xfs_healer_start.c          |  372 ++++++++++++++++++++++++++++++++++++
 healer/xfs_healer_start.service.in |   85 ++++++++
 include/builddefs.in               |    7 +
 m4/package_libcdev.m4              |   78 ++++++++
 7 files changed, 583 insertions(+), 14 deletions(-)
 create mode 100644 healer/xfs_healer_start.c
 create mode 100644 healer/xfs_healer_start.service.in
libfrog/systemd.h +15 -8
index c96df4afa39aa6..8a0970282d1080 100644
@@ -22,6 +22,20 @@ static inline bool systemd_is_service(void)
 	return getenv("SERVICE_MODE") != NULL;
 }
 
+/* Special processing for a service/daemon program that is exiting. */
+static inline int
+systemd_service_exit_now(int ret)
+{
+	/*
+	 * If we're being run as a service, the return code must fit the LSB
+	 * init script action error guidelines, which is to say that we
+	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
+	 * section 22.2) and hope the admin will scan the log for what actually
+	 * happened.
+	 */
+	return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+}
+
 /* Special processing for a service/daemon program that is exiting. */
 static inline int
 systemd_service_exit(int ret)
@@ -35,14 +49,7 @@ systemd_service_exit(int ret)
 	 */
 	sleep(2);
 
-	/*
-	 * If we're being run as a service, the return code must fit the LSB
-	 * init script action error guidelines, which is to say that we
-	 * compress all errors to 1 ("generic or unspecified error", LSB 5.0
-	 * section 22.2) and hope the admin will scan the log for what actually
-	 * happened.
-	 */
-	return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+	return systemd_service_exit_now(ret);
 }
 
 #endif /* __LIBFROG_SYSTEMD_H__ */
configure.ac +5 -0
index 78bb87b159b10b..fb1e1973d2f559 100644
@@ -189,6 +189,11 @@ AC_HAVE_BLKID_TOPO
 AC_HAVE_TRIVIAL_AUTO_VAR_INIT
 AC_STRERROR_R_RETURNS_STRING
 AC_HAVE_CLOSE_RANGE
+AC_HAVE_LISTMOUNT
+if test "$have_listmount" = "yes"; then
+	AC_HAVE_LISTMOUNT_NS_FD
+fi
+AC_HAVE_FANOTIFY_MOUNTINFO
 
 if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then
         AC_PACKAGE_CHECK_UBSAN
healer/Makefile +21 -6
index 86d6f50781f9b6..53cc787c6fcd0c 100644
@@ -9,6 +9,7 @@ include $(builddefs)
 INSTALL_HEALER = install-healer
 
 LTCOMMAND = xfs_healer
+BUILD_TARGETS = $(LTCOMMAND)
 
 CFILES = \
 fsrepair.c \
@@ -24,13 +25,27 @@ LLDFLAGS = -static
 
 ifeq ($(HAVE_SYSTEMD),yes)
 INSTALL_HEALER += install-systemd
-SYSTEMD_SERVICES=\
[email protected]
+SYSTEMD_SERVICES = \
 	system-xfs_healer.slice \
-	[email protected]
-OPTIONAL_TARGETS += $(SYSTEMD_SERVICES)
-endif
+	$(XFS_HEALER_SVCNAME)
+endif # HAVE_SYSTEMD
 
-default: depend $(LTCOMMAND) $(SYSTEMD_SERVICES)
+ifeq ($(HAVE_HEALER_START_DEPS),yes)
+CFLAGS += -DXFS_HEALER_SVCNAME=\"$(XFS_HEALER_SVCNAME)\"
+ ifeq ($(HAVE_LISTMOUNT_NS_FD),yes)
+  CFLAGS += -DHAVE_LISTMOUNT_NS_FD
+ endif # listmount mnt_ns_fd
+
+BUILD_TARGETS += xfs_healer_start
+SYSTEMD_SERVICES += xfs_healer_start.service
+endif # xfs_healer_start deps
+
+default: depend $(BUILD_TARGETS) $(SYSTEMD_SERVICES)
+
+xfs_healer_start: $(SUBDIRS) xfs_healer_start.o $(LTDEPENDENCIES)
+	@echo "    [LD]     $@"
+	$(Q)$(LTLINK) -o $@ $(LDFLAGS) xfs_healer_start.o $(LDLIBS)
 
 %.service: %.service.in $(builddefs)
 	@echo "    [SED]    $@"
@@ -43,7 +58,7 @@ install: $(INSTALL_HEALER)
 
 install-healer: default
 	$(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR)
-	$(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR)
+	$(INSTALL) -m 755 $(BUILD_TARGETS) $(PKG_LIBEXEC_DIR)
 
 install-systemd: default
 	$(INSTALL) -m 755 -d $(SYSTEMD_SYSTEM_UNIT_DIR)
healer/xfs_healer_start.c new file +372 -0

372 lines not on this page · raw message

healer/xfs_healer_start.service.in new file +85 -0

85 lines not on this page · raw message

include/builddefs.in +7 -0
index 99373ec86215cf..51d24dd854bc17 100644
@@ -120,6 +120,9 @@ UDEV_RULE_DIR = @udev_rule_dir@
 HAVE_LIBURCU_ATOMIC64 = @have_liburcu_atomic64@
 STRERROR_R_RETURNS_STRING = @strerror_r_returns_string@
 HAVE_CLOSE_RANGE = @have_close_range@
+HAVE_LISTMOUNT = @have_listmount@
+HAVE_LISTMOUNT_NS_FD = @have_listmount_ns_fd@
+HAVE_FANOTIFY_MOUNTINFO = @have_fanotify_mountinfo@
 
 GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall
 #	   -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
@@ -152,6 +155,10 @@ ifeq ($(HAVE_LIBURCU_ATOMIC64),yes)
 PCFLAGS += -DHAVE_LIBURCU_ATOMIC64
 endif
 
+ifeq ($(ENABLE_HEALER)$(HAVE_SYSTEMD)$(HAVE_LISTMOUNT)$(HAVE_FANOTIFY_MOUNTINFO),yesyesyesyes)
+HAVE_HEALER_START_DEPS = yes
+endif
+
 SANITIZER_CFLAGS += @addrsan_cflags@ @threadsan_cflags@ @ubsan_cflags@ @autovar_init_cflags@
 SANITIZER_LDFLAGS += @addrsan_ldflags@ @threadsan_ldflags@ @ubsan_ldflags@
 
m4/package_libcdev.m4 +78 -0

81 lines not on this page · raw message

#30 lore raw

Re: [PATCH 13/26] xfs_healer: create a service to start the per-mount healer service

Quoted 9 lines
healer/xfs_healer_start.c new file +9 -0
+/* Start healer services for existing XFS mounts. */
+static int
+start_existing_mounts(
+	int			mnt_ns_fd)
+{
+	struct mnt_id_req	req = {
+		.size		= sizeof(struct mnt_id_req),
+#ifdef HAVE_LISTMOUNT_NS_FD
+		.mnt_ns_fd	= mnt_ns_fd,

healer/xfs_healer_start.c line 214

Quoted 10 lines
healer/xfs_healer_start.c new file +8 -0
+#else
+		.spare		= mnt_ns_fd,
+#endif
+		.mnt_id		= LSMT_ROOT,
+	};
+	uint64_t		mnt_ids[32];
+	int			i;
 
 
+	while ((ret = syscall(SYS_listmount, &req, &mnt_ids, 32, 0)) > 0) {

Should this use a wrapper so we can switch to the type safe libc version once it becomes available?

#31 lore raw

Re: [PATCH 13/26] xfs_healer: create a service to start the per-mount healer service

Quoted 18 lines · Christoph Hellwig
Quoted 16 lines
healer/xfs_healer_start.c new file +16 -0
+/* Start healer services for existing XFS mounts. */
+static int
+start_existing_mounts(
+	int			mnt_ns_fd)
+{
+	struct mnt_id_req	req = {
+		.size		= sizeof(struct mnt_id_req),
+#ifdef HAVE_LISTMOUNT_NS_FD
+		.mnt_ns_fd	= mnt_ns_fd,
+#else
+		.spare		= mnt_ns_fd,
+#endif
+		.mnt_id		= LSMT_ROOT,
+	};
+	uint64_t		mnt_ids[32];
+	int			i;
Quoted 4 lines

healer/xfs_healer_start.c line 214

Quoted 1 line
healer/xfs_healer_start.c new file +1 -0
+	while ((ret = syscall(SYS_listmount, &req, &mnt_ids, 32, 0)) > 0) {

Should this use a wrapper so we can switch to the type safe libc version once it becomes available?

What kind of wrapper?

static inline void
set_mnt_id_req_ns_fd(struct mnt_id_req *r, int mnt_ns_fd) {

#ifdef HAVE_LISTMOUNT_NS_FD
	r->mnt_ns_fd = mnt_ns_fd;
#else
	r->spare = mnt_ns_fd;

#endif
}

or did you have something else in mind? The manual page for listmount says that glibc provides no wrapper[1].

--D

[1] https://www.man7.org/linux//man-pages/man2/listmount.2.html

#32 lore raw

Re: [PATCH 13/26] xfs_healer: create a service to start the per-mount healer service

Quoted 6 lines · Darrick J. Wong
Quoted 4 lines

healer/xfs_healer_start.c line 214

Quoted 1 line
healer/xfs_healer_start.c new file +1 -0
+	while ((ret = syscall(SYS_listmount, &req, &mnt_ids, 32, 0)) > 0) {

Should this use a wrapper so we can switch to the type safe libc version once it becomes available?

What kind of wrapper?

For calling the listmount system call.

Quoted 2 lines

or did you have something else in mind? The manual page for listmount says that glibc provides no wrapper[1].

Ånd there are no plans to provide one? :( Even if so having a libfrog wrapper would be nice rather than open coding syscall() in at least two places in this series.

#33 lore raw

Re: [PATCH 13/26] xfs_healer: create a service to start the per-mount healer service

Quoted 8 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
Quoted 4 lines

healer/xfs_healer_start.c line 214

Quoted 1 line
healer/xfs_healer_start.c new file +1 -0
+	while ((ret = syscall(SYS_listmount, &req, &mnt_ids, 32, 0)) > 0) {

Should this use a wrapper so we can switch to the type safe libc version once it becomes available?

What kind of wrapper?

Quoted 8 lines

For calling the listmount system call.

Quoted 2 lines

or did you have something else in mind? The manual page for listmount says that glibc provides no wrapper[1].

Ånd there are no plans to provide one? :( Even if so having a libfrog wrapper would be nice rather than open coding syscall() in at least two places in this series.

Oh, I see. Yes, I could create a libfrog helper to wrap the listmount callsites.

I can't tell what sorts of discussions glibc may or may not have had because sourceware is barely reachable due to AIDDOS attacks or whatever the reason du jour is, and given that the archives are pipermail they're probably not searchable anyway. :(

Google, FWIW, shows a discussion from November 2023 that seems to have dried up, and the glibc gitweb doesn't produce any hits for listmount or statmount.

So my guess is that we can just make our own libfrog wrapper and if libc support ever shows up we can always port.

--D

#34 lore raw

PATCH 14/26 xfs_healer: don't start service if kernel support unavailable

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use ExecCondition= in the system service to check if kernel support for the health monitor is available. If not, we don't want to run the service, have it fail, and generate a bunch of silly log messages.

---
 healer/xfs_healer.h           |    1 +
 healer/xfs_healer.c           |   55 +++++++++++++++++++++++++++++++----------
 healer/[email protected] |    1 +
 3 files changed, 43 insertions(+), 14 deletions(-)
healer/xfs_healer.h +1 -0
index 6d12921245934c..93cca394e9fdd1 100644
@@ -26,6 +26,7 @@ struct healer_ctx {
 	int			everything;
 	int			foreground;
 	int			want_repair;
+	int			support_check;
 
 	/* fd and fs geometry for mount */
 	struct xfs_fd		mnt;
healer/xfs_healer.c +41 -14

128 lines not on this page · raw message

healer/[email protected] +1 -0
index 385257872b0cbb..53f89cf9c4333d 100644
@@ -17,6 +17,7 @@ RequiresMountsFor=%f
 [Service]
 Type=exec
 Environment=SERVICE_MODE=1
+ExecCondition=@pkg_libexec_dir@/xfs_healer --supported %f
 ExecStart=@pkg_libexec_dir@/xfs_healer %f
 SyslogIdentifier=%N
 
#36 lore raw

PATCH 15/26 xfs_healer: use the autofsck fsproperty to select mode

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make the xfs_healer background service query the autofsck filesystem property to figure out which operating mode it should use.

---
 healer/xfs_healer.h    |    1 
 libfrog/fsproperties.h |    5 ++
 healer/xfs_healer.c    |  102 +++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 105 insertions(+), 3 deletions(-)
healer/xfs_healer.h +1 -0
index 93cca394e9fdd1..3a8f9a67beb7b6 100644
@@ -27,6 +27,7 @@ struct healer_ctx {
 	int			foreground;
 	int			want_repair;
 	int			support_check;
+	int			autofsck;
 
 	/* fd and fs geometry for mount */
 	struct xfs_fd		mnt;
libfrog/fsproperties.h +5 -0
index 11d6530bc9a6d6..1cf90d058765b2 100644
@@ -52,6 +52,11 @@ bool fsprop_validate(const char *name, const char *value);
 
 #define FSPROP_AUTOFSCK_NAME		"autofsck"
 
+/* filesystem property name for fgetxattr */
+#define VFS_FSPROP_AUTOFSCK_NAME	(FSPROP_NAMESPACE \
+					 FSPROP_NAME_PREFIX \
+					 FSPROP_AUTOFSCK_NAME)
+
 enum fsprop_autofsck {
 	FSPROP_AUTOFSCK_UNSET = 0,	/* do not set property */
 	FSPROP_AUTOFSCK_NONE,		/* no background scrubs */
healer/xfs_healer.c +99 -3

162 lines not on this page · raw message

#38 lore raw

PATCH 16/26 xfs_healer: run full scrub after lost corruption events or targeted repair failure

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If we fail to perform a spot repair of metadata or the kernel tells us that it lost corruption events due to queue limits, initiate a full run of the online fsck service to try to fix the error.

---
 healer/xfs_healer.h  |    3 ++
 healer/Makefile      |    2 +
 healer/fsrepair.c    |   81 +++++++++++++++++++++++++++++++++++++++++++++-----
 healer/weakhandle.c  |   13 ++++++++
 healer/xfs_healer.c  |    7 ++++
 include/builddefs.in |    1 +
 scrub/Makefile       |    7 ++--
 7 files changed, 102 insertions(+), 12 deletions(-)
healer/xfs_healer.h +3 -0
index 3a8f9a67beb7b6..5e9fd7fec904ab 100644
@@ -71,6 +71,7 @@ void lookup_path(struct healer_ctx *ctx,
 int repair_metadata(struct healer_ctx *ctx, const struct hme_prefix *pfx,
 		const struct xfs_health_monitor_event *hme);
 bool healer_can_repair(struct healer_ctx *ctx);
+void run_full_repair(struct healer_ctx *ctx);
 
 /* weakhandle.c */
 int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
@@ -79,5 +80,7 @@ int weakhandle_reopen(struct weakhandle *wh, int *fd);
 void weakhandle_free(struct weakhandle **whp);
 int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
 		char *path, size_t pathlen);
+int weakhandle_instance_unit_name(struct weakhandle *wh, const char *template,
+		char *unitname, size_t unitnamelen);
 
 #endif /* XFS_HEALER_XFS_HEALER_H_ */
healer/Makefile +2 -0
index 53cc787c6fcd0c..f7ee911fe11f92 100644
@@ -19,6 +19,8 @@ xfs_healer.c
 HFILES = \
 xfs_healer.h
 
+CFLAGS+=-DXFS_SCRUB_SVCNAME=\"$(XFS_SCRUB_SVCNAME)\"
+
 LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD)
 LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG)
 LLDFLAGS = -static
healer/fsrepair.c +73 -8

156 lines not on this page · raw message

healer/weakhandle.c +13 -0
index 8950e0eb1e5a43..849aa2882700d4 100644
@@ -13,6 +13,7 @@
 #include "libfrog/workqueue.h"
 #include "libfrog/getparents.h"
 #include "libfrog/paths.h"
+#include "libfrog/systemd.h"
 #include "xfs_healer.h"
 
 struct weakhandle {
@@ -199,3 +200,15 @@ weakhandle_getpath_for(
 	close(mnt_fd);
 	return ret;
 }
+
+/* Compute the systemd instance unit name for this mountpoint. */
+int
+weakhandle_instance_unit_name(
+	struct weakhandle	*wh,
+	const char		*template,
+	char			*unitname,
+	size_t			unitnamelen)
+{
+	return systemd_path_instance_unit_name(template, wh->mntpoint,
+			unitname, unitnamelen);
+}
healer/xfs_healer.c +7 -0
index 975d28789d5e14..2901ed0bbe219e 100644
@@ -138,6 +138,13 @@ handle_event(
 		pthread_mutex_unlock(&ctx->conlock);
 	}
 
+	/*
+	 * If we didn't ask for all the metadata reports (including the healthy
+	 * ones) and the kernel tells us it lost something, run the full scan.
+	 */
+	if (hme->type == XFS_HEALTH_MONITOR_TYPE_LOST && !ctx->everything)
+		run_full_repair(ctx);
+
 	/* Initiate a repair if appropriate. */
 	if (will_repair)
 		repair_metadata(ctx, &pfx, hme);
include/builddefs.in +1 -0
index 51d24dd854bc17..b5ace90f53a46e 100644
@@ -62,6 +62,7 @@ MKFS_CFG_DIR	= @datadir@/@pkg_name@/mkfs
 PKG_STATE_DIR	= @localstatedir@/lib/@pkg_name@
 
 XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP=$(PKG_STATE_DIR)/xfs_scrub_all_media.stamp
[email protected]
 
 CC		= @cc@
 BUILD_CC	= @BUILD_CC@
scrub/Makefile +3 -4
index ff79a265762332..aee49bfce100e2 100644
@@ -8,7 +8,6 @@ include $(builddefs)
 
 SCRUB_PREREQS=$(HAVE_GETFSMAP)
 
[email protected]
 [email protected]
 
 ifeq ($(SCRUB_PREREQS),yes)
@@ -21,7 +20,7 @@ XFS_SCRUB_SERVICE_ARGS = -b -o autofsck
 ifeq ($(HAVE_SYSTEMD),yes)
 INSTALL_SCRUB += install-systemd
 SYSTEMD_SERVICES=\
-	$(scrub_svcname) \
+	$(XFS_SCRUB_SVCNAME) \
 	[email protected] \
 	$(scrub_media_svcname) \
 	[email protected] \
@@ -123,7 +122,7 @@ xfs_scrub_all.timer: xfs_scrub_all.timer.in $(builddefs)
 $(XFS_SCRUB_ALL_PROG): $(XFS_SCRUB_ALL_PROG).in $(builddefs) $(TOPDIR)/libfrog/gettext.py
 	@echo "    [SED]    $@"
 	$(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \
-		   -e "s|@scrub_svcname@|$(scrub_svcname)|g" \
+		   -e "s|@scrub_svcname@|$(XFS_SCRUB_SVCNAME)|g" \
 		   -e "s|@scrub_media_svcname@|$(scrub_media_svcname)|g" \
 		   -e "s|@pkg_version@|$(PKG_VERSION)|g" \
 		   -e "s|@stampfile@|$(XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP)|g" \
@@ -137,7 +136,7 @@ $(XFS_SCRUB_ALL_PROG): $(XFS_SCRUB_ALL_PROG).in $(builddefs) $(TOPDIR)/libfrog/g
 xfs_scrub_fail: xfs_scrub_fail.in $(builddefs)
 	@echo "    [SED]    $@"
 	$(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \
-		   -e "s|@scrub_svcname@|$(scrub_svcname)|g" \
+		   -e "s|@scrub_svcname@|$(XFS_SCRUB_SVCNAME)|g" \
 		   -e "s|@pkg_version@|$(PKG_VERSION)|g"  < $< > $@
 	$(Q)chmod a+x $@
 
#40 lore raw

PATCH 17/26 xfs_healer: use getmntent to find moved filesystems

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

It's possible that a mounted filesystem can move mountpoints between the time of the initial mount (at which point xfs_healer starts) and when it actually wants to start a repair. When this happens, weakhandle::mountpoint becomes obsolete and opening it will either fail with ENOENT or the handle revalidation will return ESTALE.

However, we do still have a means to find the mounted filesystem -- the fsname parameter (aka the path to the data device at mount time). This is record in /proc/mounts, which means that we can iterate getmntent to see if we can find the mount elsewhere.

---
 healer/weakhandle.c |   50 ++++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 46 insertions(+), 4 deletions(-)
healer/weakhandle.c +46 -4
index 849aa2882700d4..8ca4ef847188ba 100644
@@ -65,10 +65,14 @@ weakhandle_alloc(
 	return -1;
 }
 
-/* Reopen a file handle obtained via weak reference. */
-int
-weakhandle_reopen(
+/*
+ * Reopen a file handle obtained via weak reference, using the given path to a
+ * mount point.
+ */
+static int
+weakhandle_reopen_from(
 	struct weakhandle	*wh,
+	const char		*path,
 	int			*fd)
 {
 	void			*hanp;
@@ -78,7 +82,7 @@ weakhandle_reopen(
 
 	*fd = -1;
 
-	mnt_fd = open(wh->mntpoint, O_RDONLY);
+	mnt_fd = open(path, O_RDONLY);
 	if (mnt_fd < 0)
 		return -1;
 
@@ -102,6 +106,44 @@ weakhandle_reopen(
 	return -1;
 }
 
+/* Reopen a file handle obtained via weak reference. */
+int
+weakhandle_reopen(
+	struct weakhandle	*wh,
+	int			*fd)
+{
+	FILE			*mtab;
+	struct mntent		*mount;
+	int			ret;
+
+	ret = weakhandle_reopen_from(wh, wh->mntpoint, fd);
+	if (!ret)
+		return 0;
+
+	mtab = setmntent(_PATH_PROC_MOUNTS, "r");
+	if (!mtab)
+		return -1;
+
+	while ((mount = getmntent(mtab)) != NULL) {
+		if (strcmp(mount->mnt_type, "xfs"))
+			continue;
+		if (strcmp(mount->mnt_fsname, wh->fsname))
+			continue;
+
+		ret = weakhandle_reopen_from(wh, mount->mnt_dir, fd);
+		if (!ret)
+			break;
+	}
+
+	if (*fd < 0) {
+		errno = ESTALE;
+		ret = -1;
+	}
+
+	endmntent(mtab);
+	return ret;
+}
+
 /* Tear down a weak handle */
 void
 weakhandle_free(
#42 lore raw

Re: [PATCH 17/26] xfs_healer: use getmntent to find moved filesystems

Quoted 6 lines · Christoph Hellwig

Looks good:

but in a way just grabing a weak handle at mount time and never dropping it would seem more useful?

Er... that /is/ what xfs_healer does -- it opens the rootdir, creates the weakhandle, the weakhandle makes a copy of the rootdir handle, and then xfs_healer closes the rootdir fd.

Later when xfs_healer needs to do a repair, it asks the weakhandle to reopen the old mountpoint path, compare that fd's handle to the sample in the weakhandle, and tell us if the fd matches.

Or did you mean that xfs_healer should keep the rootdir fd open for the duration of its existence, that way weakhandle reconnection is trivial?

[from the next patch]

Quoted 6 lines
Quoted 3 lines

When xfs_healer reopens a mountpoint to perform a repair, it should validate that the opened fd points to a file on the same filesystem as the one being monitored.

.. and if we'd always keep the week handle around we would not need this?

The trouble with keeping the rootdir fd around is that now we pin the mount and nobody can unmount the disk until they manually kill xfs_healer. IOWs, struct weakhandle is basically a wrapper around struct xfs_handle with some cleverness to avoid maintaining an open fd to the xfs filesystem when it's not needed.

(As opposed to libhandle, which maintains an open fd to the xfs filesystem until you kill the program.)

<shrug> I might not be understanding the questions though.

--D

#43 lore raw

Re: [PATCH 17/26] xfs_healer: use getmntent to find moved filesystems

Quoted 9 lines · Darrick J. Wong

Or did you mean that xfs_healer should keep the rootdir fd open for the duration of its existence, that way weakhandle reconnection is trivial?

[from the next patch]

Quoted 4 lines
Quoted 3 lines

When xfs_healer reopens a mountpoint to perform a repair, it should validate that the opened fd points to a file on the same filesystem as the one being monitored.

Quoted 8 lines
Quoted 2 lines

.. and if we'd always keep the week handle around we would not need this?

The trouble with keeping the rootdir fd around is that now we pin the mount and nobody can unmount the disk until they manually kill xfs_healer. IOWs, struct weakhandle is basically a wrapper around struct xfs_handle with some cleverness to avoid maintaining an open fd to the xfs filesystem when it's not needed.

Ok. I've officially forgot what all the kernel code did. I somehow expected a weak handle to be a fd that the kernel could close on us, which would be much more handy here.

#44 lore raw

Re: [PATCH 17/26] xfs_healer: use getmntent to find moved filesystems

Quoted 19 lines · Christoph Hellwig
Quoted 9 lines · Darrick J. Wong

Or did you mean that xfs_healer should keep the rootdir fd open for the duration of its existence, that way weakhandle reconnection is trivial?

[from the next patch]

Quoted 4 lines
Quoted 3 lines

When xfs_healer reopens a mountpoint to perform a repair, it should validate that the opened fd points to a file on the same filesystem as the one being monitored.

Quoted 8 lines
Quoted 2 lines

.. and if we'd always keep the week handle around we would not need this?

The trouble with keeping the rootdir fd around is that now we pin the mount and nobody can unmount the disk until they manually kill xfs_healer. IOWs, struct weakhandle is basically a wrapper around struct xfs_handle with some cleverness to avoid maintaining an open fd to the xfs filesystem when it's not needed.

Quoted 3 lines

Ok. I've officially forgot what all the kernel code did. I somehow expected a weak handle to be a fd that the kernel could close on us, which would be much more handy here.

Yeah. I tried creating a(nother) anon_inode that has the same sort of weak link to the xfs_mount that the healthmon fd has, for the purpose of forwarding scrub ioctls. That got twisty fast because the scrub code wants to be able to call things like mnt_want_write_file and file_inode, but the file doesn't point to an xfs inode and abusing the anon inode file to make that work just became too gross to stomach. :/

--D

#45 lore raw

Re: [PATCH 17/26] xfs_healer: use getmntent to find moved filesystems

Quoted 6 lines · Darrick J. Wong

Yeah. I tried creating a(nother) anon_inode that has the same sort of weak link to the xfs_mount that the healthmon fd has, for the purpose of forwarding scrub ioctls. That got twisty fast because the scrub code wants to be able to call things like mnt_want_write_file and file_inode, but the file doesn't point to an xfs inode and abusing the anon inode file to make that work just became too gross to stomach. :/

Yeah, don't see how that work. I guess IFF we wanted that it would have to be a VFS-level weak FD concept, and that seems out of scope for this at the moment unfortunately.

#46 lore raw

Re: [PATCH 17/26] xfs_healer: use getmntent to find moved filesystems

Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong

Yeah. I tried creating a(nother) anon_inode that has the same sort of weak link to the xfs_mount that the healthmon fd has, for the purpose of forwarding scrub ioctls. That got twisty fast because the scrub code wants to be able to call things like mnt_want_write_file and file_inode, but the file doesn't point to an xfs inode and abusing the anon inode file to make that work just became too gross to stomach. :/

Yeah, don't see how that work. I guess IFF we wanted that it would have to be a VFS-level weak FD concept, and that seems out of scope for this at the moment unfortunately.

<nod> I'll add a comment summarizing this part of the discussion to the commit message for adding repair functionality, so that we record the justification for all the getmntent trickery.

/me observes that 7.0 adds a STATMOUNT_BY_FD flag to statmount() that enables us to get the statmount data (and hence mnt_id) for an open file. We could record that (in addition to the datadev path) to try to reconnect after a mount --move without having to parse /proc/mtab like getmntent does.

--D

#47 lore raw

PATCH 18/26 xfs_healer: validate that repair fds point to the monitored fs

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When xfs_healer reopens a mountpoint to perform a repair, it should validate that the opened fd points to a file on the same filesystem as the one being monitored.

---
 healer/xfs_healer.h |    4 +++-
 healer/fsrepair.c   |   18 +++++++++++++++++-
 healer/weakhandle.c |   20 +++++++++++++++-----
 3 files changed, 35 insertions(+), 7 deletions(-)
healer/xfs_healer.h +3 -1
index 5e9fd7fec904ab..d613c4f65fe9eb 100644
@@ -76,7 +76,9 @@ void run_full_repair(struct healer_ctx *ctx);
 /* weakhandle.c */
 int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
 		struct weakhandle **whp);
-int weakhandle_reopen(struct weakhandle *wh, int *fd);
+typedef bool (*weakhandle_fd_t)(int mnt_fd, void *data);
+int weakhandle_reopen(struct weakhandle *wh, int *fd,
+		weakhandle_fd_t is_acceptable, void *data);
 void weakhandle_free(struct weakhandle **whp);
 int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
 		char *path, size_t pathlen);
healer/fsrepair.c +17 -1
index 9f8c128e395ebc..002e5e78fcf22e 100644
@@ -233,6 +233,22 @@ try_repair_inode(
 	return REPAIR_DONE;
 }
 
+/* Make sure the reopened file is on the same fs as the monitor. */
+static bool
+is_same_fs(
+	int				mnt_fd,
+	void				*data)
+{
+	struct xfs_health_file_on_monitored_fs hms = {
+		.fd = mnt_fd,
+	};
+	FILE				*mon_fp = data;
+	int				ret;
+
+	ret = ioctl(fileno(mon_fp), XFS_IOC_HEALTH_FD_ON_MONITORED_FS, &hms);
+	return ret == 0;
+}
+
 /* Repair a metadata corruption. */
 int
 repair_metadata(
@@ -244,7 +260,7 @@ repair_metadata(
 	int					repair_fd;
 	int					ret;
 
-	ret = weakhandle_reopen(ctx->wh, &repair_fd);
+	ret = weakhandle_reopen(ctx->wh, &repair_fd, is_same_fs, ctx->mon_fp);
 	if (ret) {
 		fprintf(stderr, "%s: %s: %s\n", ctx->mntpoint,
 				_("cannot open filesystem to repair"),
healer/weakhandle.c +15 -5
index 8ca4ef847188ba..4b0e2e991702ca 100644
@@ -73,7 +73,9 @@ static int
 weakhandle_reopen_from(
 	struct weakhandle	*wh,
 	const char		*path,
-	int			*fd)
+	int			*fd,
+	weakhandle_fd_t		is_acceptable,
+	void			*data)
 {
 	void			*hanp;
 	size_t			hlen;
@@ -95,6 +97,11 @@ weakhandle_reopen_from(
 		goto out_handle;
 	}
 
+	if (is_acceptable && !is_acceptable(mnt_fd, data)) {
+		errno = ESTALE;
+		goto out_handle;
+	}
+
 	free_handle(hanp, hlen);
 	*fd = mnt_fd;
 	return 0;
@@ -110,13 +117,15 @@ weakhandle_reopen_from(
 int
 weakhandle_reopen(
 	struct weakhandle	*wh,
-	int			*fd)
+	int			*fd,
+	weakhandle_fd_t		is_acceptable,
+	void			*data)
 {
 	FILE			*mtab;
 	struct mntent		*mount;
 	int			ret;
 
-	ret = weakhandle_reopen_from(wh, wh->mntpoint, fd);
+	ret = weakhandle_reopen_from(wh, wh->mntpoint, fd, is_acceptable, data);
 	if (!ret)
 		return 0;
 
@@ -130,7 +139,8 @@ weakhandle_reopen(
 		if (strcmp(mount->mnt_fsname, wh->fsname))
 			continue;
 
-		ret = weakhandle_reopen_from(wh, mount->mnt_dir, fd);
+		ret = weakhandle_reopen_from(wh, mount->mnt_dir, fd,
+				is_acceptable, data);
 		if (!ret)
 			break;
 	}
@@ -215,7 +225,7 @@ weakhandle_getpath_for(
 	fakehandle.ha_fid.fid_ino = ino;
 	fakehandle.ha_fid.fid_gen = gen;
 
-	ret = weakhandle_reopen(wh, &mnt_fd);
+	ret = weakhandle_reopen(wh, &mnt_fd, NULL, NULL);
 	if (ret)
 		return ret;
 
#48 lore raw

Re: [PATCH 18/26] xfs_healer: validate that repair fds point to the monitored fs

Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When xfs_healer reopens a mountpoint to perform a repair, it should validate that the opened fd points to a file on the same filesystem as the one being monitored.

.. and if we'd always keep the week handle around we would not need this?

#49 lore raw

PATCH 19/26 xfs_healer: add a manual page

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a new section 8 manpage for this service daemon so others can read about what this program is supposed to do.

---
 man/man8/Makefile           |   40 +++++++++++++---
 man/man8/xfs_healer.8       |  109 +++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_healer_start.8 |   37 +++++++++++++++
 3 files changed, 180 insertions(+), 6 deletions(-)
 create mode 100644 man/man8/xfs_healer.8
 create mode 100644 man/man8/xfs_healer_start.8
man/man8/Makefile +34 -6
index 5be76ab727a1fe..05710f85ae89ad 100644
@@ -7,13 +7,41 @@ include $(TOPDIR)/include/builddefs
 
 MAN_SECTION	= 8
 
-ifneq ("$(ENABLE_SCRUB)","yes")
-  MAN_PAGES = $(filter-out xfs_scrub%,$(shell echo *.$(MAN_SECTION)))
-else
-  MAN_PAGES = $(shell echo *.$(MAN_SECTION))
-  MAN_PAGES += xfs_scrub_all.8
+MAN_PAGES = \
+	fsck.xfs.8 \
+	mkfs.xfs.8 \
+	xfs_admin.8 \
+	xfs_bmap.8 \
+	xfs_copy.8 \
+	xfs_db.8 \
+	xfs_estimate.8 \
+	xfs_freeze.8 \
+	xfs_fsr.8 \
+	xfs_growfs.8 \
+	xfs_info.8 \
+	xfs_io.8 \
+	xfs_logprint.8 \
+	xfs_mdrestore.8 \
+	xfs_metadump.8 \
+	xfs_mkfile.8 \
+	xfs_ncheck.8 \
+	xfs_property.8 \
+	xfs_protofile.8 \
+	xfs_quota.8 \
+	xfs_repair.8 \
+	xfs_rtcp.8 \
+	xfs_spaceman.8
+
+ifeq ($(ENABLE_HEALER),yes)
+  MAN_PAGES += xfs_healer.8
 endif
-MAN_PAGES	+= mkfs.xfs.8
+ifeq ($(HAVE_HEALER_START_DEPS),yes)
+  MAN_PAGES += xfs_healer_start.8
+endif
+ifeq ($(ENABLE_SCRUB),yes)
+  MAN_PAGES += xfs_scrub.8 xfs_scrub_all.8
+endif
+
 MAN_DEST	= $(PKG_MAN_DIR)/man$(MAN_SECTION)
 LSRCFILES	= $(MAN_PAGES)
 DIRT		= mkfs.xfs.8 xfs_scrub_all.8
man/man8/xfs_healer.8 new file +109 -0

109 lines not on this page · raw message

man/man8/xfs_healer_start.8 new file +37 -0
new file mode 100644
index 00000000000000..9e424432a513fe
@@ -0,0 +1,37 @@
+.TH xfs_healer_start 8
+.SH NAME
+xfs_healer_start \- starts xfs_healer instances
+.SH SYNOPSIS
+.B xfs_healer_start
+[
+.B OPTIONS
+]
+.br
+.B xfs_healer \-V
+.SH DESCRIPTION
+.B xfs_healer_start
+starts the xfs_healer service whenever the kernel mounts an XFS filesystem in
+the current mount namespace.
+.PP
+.B WARNING!
+This program is
+.BR EXPERIMENTAL ","
+which means that its behavior and interface
+could change at any time!
+
+Normally this program runs as a systemd service.
+
+.SH OPTIONS
+.TP
+.B \-\-supported
+Check if the kernel supports listening for mount events.
+Exits with 0 if it does, and non-zero if not.
+.TP
+.BI "\-\-mountns " path
+Monitor the given mount namespace.
+Defaults to the mount namespace associated with the process itself.
+.TP
+.BI \-V
+Prints the version number and exit.
+.SH SEE ALSO
+.BR xfs_healer (8).
#51 lore raw

PATCH 20/26 xfs_scrub: use the verify media ioctl during phase 6 if possible

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the kernel suppots the XFS_IOC_VERIFY_MEDIA ioctl, use that to perform the phase 6 media scan instead of pwrite or the SCSI VERIFY command. This enables better integration with xfs_healer and fsnotify; and reduces the amount of work that userspace has to do.

---
 scrub/disk.h        |   11 ++++++++++-
 scrub/disk.c        |   40 +++++++++++++++++++++++++++++++++++++++-
 scrub/phase1.c      |   25 +++++++++++++++++++++++++
 scrub/read_verify.c |    2 +-
 4 files changed, 75 insertions(+), 3 deletions(-)
scrub/disk.h +10 -1
index 73c73ab57fb5c7..2ae27b73839ad3 100644
@@ -10,18 +10,27 @@
 struct disk {
 	struct stat	d_sb;
 	int		d_fd;
+	int		d_verify_fd;
 	unsigned int	d_lbalog;
 	unsigned int	d_lbasize;	/* bytes */
 	unsigned int	d_flags;
 	unsigned int	d_blksize;	/* bytes */
 	uint64_t	d_size;		/* bytes */
 	uint64_t	d_start;	/* bytes */
+	unsigned int	d_verify_disk;
 };
 
 unsigned int disk_heads(struct disk *disk);
 struct disk *disk_open(const char *pathname);
 int disk_close(struct disk *disk);
 ssize_t disk_read_verify(struct disk *disk, void *buf, uint64_t startblock,
-		uint64_t blockcount);
+		uint64_t blockcount, bool single_step);
+
+static inline void
+disk_config_xfs_verify(struct disk *disk, int mnt_fd, unsigned int verify_disk)
+{
+	disk->d_verify_fd = mnt_fd;
+	disk->d_verify_disk = verify_disk;
+}
 
 #endif /* XFS_SCRUB_DISK_H_ */
scrub/disk.c +39 -1
index 2cf84d91887587..4e78bd1cebbdc8 100644
@@ -190,6 +190,7 @@ disk_open(
 	disk = calloc(1, sizeof(struct disk));
 	if (!disk)
 		return NULL;
+	disk->d_verify_fd = -1;
 
 	disk->d_fd = open(pathname, O_RDONLY | O_DIRECT | O_NOATIME);
 	if (disk->d_fd < 0)
@@ -266,6 +267,18 @@ disk_close(
 #define LBASIZE(d)		(1ULL << (d)->d_lbalog)
 #define BTOLBA(d, bytes)	(((uint64_t)(bytes) + LBASIZE(d) - 1) >> (d)->d_lbalog)
 
+#ifndef BTOBB
+# define BTOBB(bytes)		((uint64_t)((bytes) + 511) >> 9)
+#endif
+
+#ifndef BTOBBT
+# define BTOBBT(bytes)		((uint64_t)(bytes) >> 9)
+#endif
+
+#ifndef BBTOB
+# define BBTOB(bytes)		((uint64_t)(bytes) << 9)
+#endif
+
 /* Simulate disk errors. */
 static int
 disk_simulate_read_error(
@@ -329,7 +342,8 @@ disk_read_verify(
 	struct disk		*disk,
 	void			*buf,
 	uint64_t		start,
-	uint64_t		length)
+	uint64_t		length,
+	bool			single_step)
 {
 	if (debug) {
 		int		ret;
@@ -345,6 +359,30 @@ disk_read_verify(
 			return length;
 	}
 
+	if (disk->d_verify_fd >= 0) {
+		const uint64_t	orig_start_daddr = BTOBBT(start);
+		struct xfs_verify_media me = {
+			.me_start_daddr	= orig_start_daddr,
+			.me_end_daddr	= BTOBB(start + length),
+			.me_dev		= disk->d_verify_disk,
+			.me_rest_us	= bg_mode > 2 ? bg_mode - 1 : 0,
+		};
+		int		ret;
+
+		if (single_step)
+			me.me_flags |= XFS_VERIFY_MEDIA_REPORT;
+
+		ret = ioctl(disk->d_verify_fd, XFS_IOC_VERIFY_MEDIA, &me);
+		if (ret < 0)
+			return ret;
+		if (me.me_ioerror) {
+			errno = me.me_ioerror;
+			return -1;
+		}
+
+		return BBTOB(me.me_start_daddr - orig_start_daddr);
+	}
+
 	/* Convert to logical block size. */
 	if (disk->d_flags & DISK_FLAG_SCSI_VERIFY)
 		return disk_scsi_verify(disk, BTOLBAT(disk, start),
scrub/phase1.c +25 -0
index 10e9aa1892b701..093e7a01b9542f 100644
@@ -213,6 +213,29 @@ mode_from_autofsck(
 	goto summarize;
 }
 
+/* Does the XFS driver support media scanning its own disks? */
+static void
+configure_xfs_verify(
+	struct scrub_ctx	*ctx)
+{
+	struct xfs_verify_media	me = {
+		.me_start_daddr	= 1,
+		.me_end_daddr	= 0,
+		.me_dev		= XFS_DEV_DATA,
+	};
+	int			ret;
+
+	ret = ioctl(ctx->mnt.fd, XFS_IOC_VERIFY_MEDIA, &me);
+	if (ret < 0)
+		return;
+
+	disk_config_xfs_verify(ctx->datadev, ctx->mnt.fd, XFS_DEV_DATA);
+	if (ctx->logdev)
+		disk_config_xfs_verify(ctx->logdev, ctx->mnt.fd, XFS_DEV_LOG);
+	if (ctx->rtdev)
+		disk_config_xfs_verify(ctx->rtdev, ctx->mnt.fd, XFS_DEV_RT);
+}
+
 /*
  * Bind to the mountpoint, read the XFS geometry, bind to the block devices.
  * Anything we've already built will be cleaned up by scrub_cleanup.
@@ -379,6 +402,8 @@ _("Unable to find realtime device path."));
 		}
 	}
 
+	configure_xfs_verify(ctx);
+
 	/*
 	 * Everything's set up, which means any failures recorded after
 	 * this point are most probably corruption errors (as opposed to
scrub/read_verify.c +1 -1
index 1219efe2590182..9e1f3ec0ed1186 100644
@@ -201,7 +201,7 @@ read_verify(
 		dbg_printf("diskverify %d %"PRIu64" %zu\n", rvp->disk->d_fd,
 				rv->io_start, len);
 		sz = disk_read_verify(rvp->disk, rvp->readbuf, rv->io_start,
-				len);
+				len, io_max_size <= rvp->miniosz);
 		if (sz == len && io_max_size < rvp->miniosz) {
 			/*
 			 * If the verify request was 100% successful and less
#52 lore raw

Re: [PATCH 20/26] xfs_scrub: use the verify media ioctl during phase 6 if possible

Quoted 6 lines
scrub/disk.c +1 -0
 	if (disk->d_fd < 0)
@@ -266,6 +267,18 @@ disk_close(
 #define LBASIZE(d)		(1ULL << (d)->d_lbalog)
 #define BTOLBA(d, bytes)	(((uint64_t)(bytes) + LBASIZE(d) - 1) >> (d)->d_lbalog)
 
+#ifndef BTOBB

scrub/disk.c disk_close( · line 280

Quoted 10 lines
scrub/disk.c +10 -0
+# define BTOBB(bytes)		((uint64_t)((bytes) + 511) >> 9)
+#endif
+
+#ifndef BTOBBT
+# define BTOBBT(bytes)		((uint64_t)(bytes) >> 9)
+#endif
+
+#ifndef BBTOB
+# define BBTOB(bytes)		((uint64_t)(bytes) << 9)
+#endif

Is this really something that should be in scrub and not in common code? And why the ifndef? the 9 and the derived from that 511 would also really benefit from symbolic names.

Quoted 13 lines
scrub/disk.c +13 -0
+	if (disk->d_verify_fd >= 0) {
+		const uint64_t	orig_start_daddr = BTOBBT(start);
+		struct xfs_verify_media me = {
+			.me_start_daddr	= orig_start_daddr,
+			.me_end_daddr	= BTOBB(start + length),
+			.me_dev		= disk->d_verify_disk,
+			.me_rest_us	= bg_mode > 2 ? bg_mode - 1 : 0,
+		};
+		int		ret;
+
+		if (single_step)
+			me.me_flags |= XFS_VERIFY_MEDIA_REPORT;
+

scrub/disk.c disk_read_verify( · line 384

Quoted 10 lines
scrub/disk.c +10 -0
+		ret = ioctl(disk->d_verify_fd, XFS_IOC_VERIFY_MEDIA, &me);
+		if (ret < 0)
+			return ret;
+		if (me.me_ioerror) {
+			errno = me.me_ioerror;
+			return -1;
+		}
+
+		return BBTOB(me.me_start_daddr - orig_start_daddr);
+	}

split this whole block into a helper for readabiltity?

#53 lore raw

Re: [PATCH 20/26] xfs_scrub: use the verify media ioctl during phase 6 if possible

Quoted 5 lines · Christoph Hellwig
Quoted 5 lines
scrub/disk.c +0 -0
 	if (disk->d_fd < 0)
@@ -266,6 +267,18 @@ disk_close(
 #define LBASIZE(d)		(1ULL << (d)->d_lbalog)
 #define BTOLBA(d, bytes)	(((uint64_t)(bytes) + LBASIZE(d) - 1) >> (d)->d_lbalog)
 
Quoted 15 lines

scrub/disk.c disk_close( · line 280

Quoted 11 lines
scrub/disk.c +11 -0
+#ifndef BTOBB
+# define BTOBB(bytes)		((uint64_t)((bytes) + 511) >> 9)
+#endif
+
+#ifndef BTOBBT
+# define BTOBBT(bytes)		((uint64_t)(bytes) >> 9)
+#endif
+
+#ifndef BBTOB
+# define BBTOB(bytes)		((uint64_t)(bytes) << 9)
+#endif

Is this really something that should be in scrub and not in common code? And why the ifndef? the 9 and the derived from that 511 would also really benefit from symbolic names.

Hrmm, that's a good question, why /did/ I duplicate that from xfs_fs.h? I have no idea why and it builds fine without it so I'll drop it.

Quoted 17 lines
Quoted 17 lines
scrub/disk.c +17 -0
+	if (disk->d_verify_fd >= 0) {
+		const uint64_t	orig_start_daddr = BTOBBT(start);
+		struct xfs_verify_media me = {
+			.me_start_daddr	= orig_start_daddr,
+			.me_end_daddr	= BTOBB(start + length),
+			.me_dev		= disk->d_verify_disk,
+			.me_rest_us	= bg_mode > 2 ? bg_mode - 1 : 0,
+		};
+		int		ret;
+
+		if (single_step)
+			me.me_flags |= XFS_VERIFY_MEDIA_REPORT;
+
+		ret = ioctl(disk->d_verify_fd, XFS_IOC_VERIFY_MEDIA, &me);
+		if (ret < 0)
+			return ret;
+		if (me.me_ioerror) {
Quoted 8 lines

scrub/disk.c disk_read_verify( · line 384

Quoted 6 lines
scrub/disk.c +6 -0
+			errno = me.me_ioerror;
+			return -1;
+		}
+
+		return BBTOB(me.me_start_daddr - orig_start_daddr);
+	}

split this whole block into a helper for readabiltity?

Will do.

--D

#54 lore raw

PATCH 21/26 xfs_scrub: perform media scanning of the log region

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Scan the log area for media errors because a defect in a region could prevent the user from being able to perform log recovery.

---
 scrub/phase6.c |   11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
scrub/phase6.c +8 -3
index abf6f9713f1a4d..59e05e8aa2f54d 100644
@@ -616,9 +616,14 @@ check_rmap(
 			map->fmr_flags);
 
 	/* "Unknown" extents should be verified; they could be data. */
-	if ((map->fmr_flags & FMR_OF_SPECIAL_OWNER) &&
-			map->fmr_owner == XFS_FMR_OWN_UNKNOWN)
-		map->fmr_flags &= ~FMR_OF_SPECIAL_OWNER;
+	if ((map->fmr_flags & FMR_OF_SPECIAL_OWNER)) {
+		switch (map->fmr_owner) {
+		case XFS_FMR_OWN_UNKNOWN:
+		case XFS_FMR_OWN_LOG:
+			map->fmr_flags &= ~FMR_OF_SPECIAL_OWNER;
+			break;
+		}
+	}
 
 	/*
 	 * We only care about read-verifying data extents that have been
Reviewed-by on v1 21/26
#55 lore raw

Re: [PATCH 21/26] xfs_scrub: perform media scanning of the log region

Quoted 4 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Scan the log area for media errors because a defect in a region could prevent the user from being able to perform log recovery.

Looks good. Not that we really could do anything here..

#56 lore raw

PATCH 22/26 xfs_io: add listmount command

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a command to list all mounts, now that we use this in xfs_healer_start.

---
 io/io.h           |    6 +
 io/Makefile       |    8 +
 io/init.c         |    1 
 io/listmount.c    |  383 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_io.8 |   43 ++++++
 5 files changed, 441 insertions(+)
 create mode 100644 io/listmount.c
io/io.h +6 -0
index 0f12b3cfed5e76..5f1f278d14a033 100644
@@ -164,3 +164,9 @@ void			fsprops_init(void);
 void			aginfo_init(void);
 void			healthmon_init(void);
 void			verifymedia_init(void);
+
+#ifdef HAVE_LISTMOUNT
+void			listmount_init(void);
+#else
+# define		listmount_init()	do { } while (0)
+#endif
io/Makefile +8 -0
index 79d5e172b8f31f..4c3359c4d4f7f4 100644
@@ -90,6 +90,14 @@ ifeq ($(HAVE_GETFSMAP),yes)
 CFILES += fsmap.c
 endif
 
+ifeq ($(HAVE_LISTMOUNT),yes)
+CFILES += listmount.c
+LCFLAGS += -DHAVE_LISTMOUNT
+ ifeq ($(HAVE_LISTMOUNT_NS_FD),yes)
+  CFLAGS += -DHAVE_LISTMOUNT_NS_FD
+ endif # listmount mnt_ns_fd
+endif
+
 default: depend $(LTCOMMAND)
 
 include $(BUILDRULES)
io/init.c +1 -0
index f2a551ef559200..ba60cb2199639b 100644
@@ -94,6 +94,7 @@ init_commands(void)
 	fsprops_init();
 	healthmon_init();
 	verifymedia_init();
+	listmount_init();
 }
 
 /*
io/listmount.c new file +383 -0

383 lines not on this page · raw message

man/man8/xfs_io.8 +43 -0

49 lines not on this page · raw message

#57 lore raw

Re: [PATCH 22/26] xfs_io: add listmount command

io/listmount.c line 15

Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a command to list all mounts, now that we use this in xfs_healer_start.

io/listmount.c new file +2 -0
+/* copied from linux/mount.h in linux 6.18 */
+struct statmount_fixed {

Shouldn't this use the kernel uapi header and/or a copy of it?

And be split out of the .c file into a separate header for maintainance and eventually nuking once the kernel requirement becomes new enough?

io/listmount.c line 140

Quoted 8 lines
io/listmount.c new file +8 -0
+static int
+listmount(
+	const struct mnt_id_req	*req,
+	uint64_t		*mnt_ids,
+	size_t			nr_mnt_ids)
+{
+	return syscall(SYS_listmount, req, mnt_ids, nr_mnt_ids, 0);
+}

Same comment as for the other listmount instance here.

io/listmount.c line 149

Quoted 9 lines
io/listmount.c new file +9 -0
+
+static int
+statmount(
+	const struct mnt_id_req	*req,
+	struct statmount_fixed	*smbuf,
+	size_t			smbuf_size)
+{
+	return syscall(SYS_statmount, req, smbuf, smbuf_size, 0);
+}

and similar here.

#58 lore raw

Re: [PATCH 22/26] xfs_io: add listmount command

Quoted 25 lines · Christoph Hellwig

io/listmount.c line 15

Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a command to list all mounts, now that we use this in xfs_healer_start.

io/listmount.c new file +2 -0
+/* copied from linux/mount.h in linux 6.18 */
+struct statmount_fixed {

Shouldn't this use the kernel uapi header and/or a copy of it?

And be split out of the .c file into a separate header for maintainance and eventually nuking once the kernel requirement becomes new enough?

io/listmount.c line 140

Quoted 8 lines
io/listmount.c new file +8 -0
+static int
+listmount(
+	const struct mnt_id_req	*req,
+	uint64_t		*mnt_ids,
+	size_t			nr_mnt_ids)
+{
+	return syscall(SYS_listmount, req, mnt_ids, nr_mnt_ids, 0);
+}

Same comment as for the other listmount instance here.

Quoted 11 lines

io/listmount.c line 149

Quoted 9 lines
io/listmount.c new file +9 -0
+
+static int
+statmount(
+	const struct mnt_id_req	*req,
+	struct statmount_fixed	*smbuf,
+	size_t			smbuf_size)
+{
+	return syscall(SYS_statmount, req, smbuf, smbuf_size, 0);
+}

and similar here.

Yeah, I'll move all this into a libfrog .c/.h file.

--D

#59 lore raw

PATCH 23/26 xfs_io: print systemd service names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add an xfs_io subcommand so that we can emit systemd service names for XFS services targetting filesystems paths instead of opencoding the computation in things like fstests.

---
 healer/Makefile      |    1 -
 include/builddefs.in |    2 +
 io/Makefile          |    4 +++
 io/scrub.c           |   75 ++++++++++++++++++++++++++++++++++++++++++++++++++
 man/man8/xfs_io.8    |   24 ++++++++++++++++
 scrub/Makefile       |    6 +---
 6 files changed, 107 insertions(+), 5 deletions(-)
healer/Makefile +0 -1
index f7ee911fe11f92..4116e338cd1dee 100644
@@ -27,7 +27,6 @@ LLDFLAGS = -static
 
 ifeq ($(HAVE_SYSTEMD),yes)
 INSTALL_HEALER += install-systemd
[email protected]
 SYSTEMD_SERVICES = \
 	system-xfs_healer.slice \
 	$(XFS_HEALER_SVCNAME)
include/builddefs.in +2 -0
index b5ace90f53a46e..439b0dbf3ea813 100644
@@ -63,6 +63,8 @@ PKG_STATE_DIR	= @localstatedir@/lib/@pkg_name@
 
 XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP=$(PKG_STATE_DIR)/xfs_scrub_all_media.stamp
 [email protected]
[email protected]
[email protected]
 
 CC		= @cc@
 BUILD_CC	= @BUILD_CC@
io/Makefile +4 -0
index 4c3359c4d4f7f4..7ac45236cc8c9c 100644
@@ -98,6 +98,10 @@ LCFLAGS += -DHAVE_LISTMOUNT
  endif # listmount mnt_ns_fd
 endif
 
+CFLAGS+=-DXFS_SCRUB_SVCNAME=\"$(XFS_SCRUB_SVCNAME)\"
+CFLAGS+=-DXFS_SCRUB_MEDIA_SVCNAME=\"$(XFS_SCRUB_MEDIA_SVCNAME)\"
+CFLAGS+=-DXFS_HEALER_SVCNAME=\"$(XFS_HEALER_SVCNAME)\"
+
 default: depend $(LTCOMMAND)
 
 include $(BUILDRULES)
io/scrub.c +75 -0

96 lines not on this page · raw message

man/man8/xfs_io.8 +24 -0
index 2b0dbfbe848bce..04bd1e6427f1c6 100644
@@ -1824,6 +1824,30 @@ .SH OTHER COMMANDS
 See the
 .B print
 command.
+.TP
+.BI "svcname [ \-h | \-m | \-s | \-t " template " ] " path
+Print the systemd service name for a given filesystem path.
+.RE
+.RS 1.0i
+.PD 0
+.TP
+.B \-h
+Print the systemd service name for xfs_healer.
+
+.TP
+.B \-m
+Print the systemd service name for xfs_scrub_media.
+
+.TP
+.B \-s
+Print the systemd service name for xfs_scrub.
+
+.TP
+.BI "\-t " template
+Print the systemd service name for the given template.
+.RE
+.PD
+
 .TP
 .B quit
 Exit
scrub/Makefile +2 -4
index aee49bfce100e2..6ace458118fc92 100644
@@ -8,8 +8,6 @@ include $(builddefs)
 
 SCRUB_PREREQS=$(HAVE_GETFSMAP)
 
[email protected]
-
 ifeq ($(SCRUB_PREREQS),yes)
 LTCOMMAND = xfs_scrub
 INSTALL_SCRUB = install-scrub
@@ -22,7 +20,7 @@ INSTALL_SCRUB += install-systemd
 SYSTEMD_SERVICES=\
 	$(XFS_SCRUB_SVCNAME) \
 	[email protected] \
-	$(scrub_media_svcname) \
+	$(XFS_SCRUB_MEDIA_SVCNAME) \
 	[email protected] \
 	xfs_scrub_all.service \
 	xfs_scrub_all_fail.service \
@@ -123,7 +121,7 @@ $(XFS_SCRUB_ALL_PROG): $(XFS_SCRUB_ALL_PROG).in $(builddefs) $(TOPDIR)/libfrog/g
 	@echo "    [SED]    $@"
 	$(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \
 		   -e "s|@scrub_svcname@|$(XFS_SCRUB_SVCNAME)|g" \
-		   -e "s|@scrub_media_svcname@|$(scrub_media_svcname)|g" \
+		   -e "s|@scrub_media_svcname@|$(XFS_SCRUB_MEDIA_SVCNAME)|g" \
 		   -e "s|@pkg_version@|$(PKG_VERSION)|g" \
 		   -e "s|@stampfile@|$(XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP)|g" \
 		   -e "s|@scrub_service_args@|$(XFS_SCRUB_SERVICE_ARGS)|g" \
#60 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

The idea of this is good, but is xfs_io really the right place for it? I'd expect scrub or healer to just output this somewhow.

#61 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

Quoted 3 lines · Christoph Hellwig

The idea of this is good, but is xfs_io really the right place for it? I'd expect scrub or healer to just output this somewhow.

I suppose they could, but I didn't want to clutter up the argv parsing any more than I had to; and xfs_healer gets installed to /usr/libexec which would make fstests' use of them to find the service name more complex.

(That was a long way of saying "can't we just keep using xfs_io as a dumping ground for QA-related xfs stuff?" ;))

--D

#62 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

Quoted 2 lines · Darrick J. Wong

(That was a long way of saying "can't we just keep using xfs_io as a dumping ground for QA-related xfs stuff?" ;))

I really hate messing it up with things that are no I/O at all, and not related to issuing I/O or related syscalls. Maybe just add a new little binary for it?

#63 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong

(That was a long way of saying "can't we just keep using xfs_io as a dumping ground for QA-related xfs stuff?" ;))

I really hate messing it up with things that are no I/O at all, and not related to issuing I/O or related syscalls. Maybe just add a new little binary for it?

How about xfs_db, since normal users shouldn't need to compute the service unit names?

--D

#64 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

Quoted 11 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong

(That was a long way of saying "can't we just keep using xfs_io as a dumping ground for QA-related xfs stuff?" ;))

I really hate messing it up with things that are no I/O at all, and not related to issuing I/O or related syscalls. Maybe just add a new little binary for it?

How about xfs_db, since normal users shouldn't need to compute the service unit names?

Still seems totally out of place for something not touching the on-disk structures. What's the problem with adding a new trivial binary for it? Or even just publishing the name in a file in /usr/share?

#65 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong

(That was a long way of saying "can't we just keep using xfs_io as a dumping ground for QA-related xfs stuff?" ;))

I really hate messing it up with things that are no I/O at all, and not related to issuing I/O or related syscalls. Maybe just add a new little binary for it?

Quoted 7 lines
Quoted 2 lines

How about xfs_db, since normal users shouldn't need to compute the service unit names?

Still seems totally out of place for something not touching the on-disk structures. What's the problem with adding a new trivial binary for it? Or even just publishing the name in a file in /usr/share?

Eh I'll just put it in xfs_{scrub,healer} as a --svcname argument.

$ xfs_scrub --svcname /home
[email protected]
$ xfs_scrub --svcname -x /home
[email protected]
$ xfs_healer --svcname /home
[email protected]

--D

#66 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

Quoted 6 lines · Darrick J. Wong
Quoted 4 lines

Still seems totally out of place for something not touching the on-disk structures. What's the problem with adding a new trivial binary for it? Or even just publishing the name in a file in /usr/share?

Eh I'll just put it in xfs_{scrub,healer} as a --svcname argument.

I think I proposed something like this before, and I really like that!

#67 lore raw

Re: [PATCH 23/26] xfs_io: print systemd service names

Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
Quoted 4 lines

Still seems totally out of place for something not touching the on-disk structures. What's the problem with adding a new trivial binary for it? Or even just publishing the name in a file in /usr/share?

Eh I'll just put it in xfs_{scrub,healer} as a --svcname argument.

I think I proposed something like this before, and I really like that!

Yes, you did propose it before, and I figured you'd like it :)

--D

#68 lore raw

PATCH 24/26 mkfs: enable online repair if all backrefs are enabled

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If all backreferences are enabled in the filesystem, then enable online repair by default if the user didn't supply any other autofsck setting. Users might as well get full self-repair capability if they're paying for the extra metadata.

---
 mkfs/xfs_mkfs.c |    9 +++++++++
 1 file changed, 9 insertions(+)
mkfs/xfs_mkfs.c +9 -0
index a11994027c2df1..87bdf0e22b96f8 100644
@@ -6289,6 +6289,15 @@ main(
 	if (mp->m_sb.sb_agcount > 1)
 		rewrite_secondary_superblocks(mp);
 
+	/*
+	 * If the filesystem has full backreferences and the user didn't
+	 * express an autofsck preference, enable online repair because they
+	 * might as well get some useful functionality from the extra metadata.
+	 */
+	if (cli.autofsck == FSPROP_AUTOFSCK_UNSET &&
+	    cli.sb_feat.rmapbt && cli.sb_feat.parent_pointers)
+		cli.autofsck = FSPROP_AUTOFSCK_REPAIR;
+
 	if (cli.autofsck != FSPROP_AUTOFSCK_UNSET)
 		set_autofsck(mp, &cli);
 
#69 lore raw

Re: [PATCH 24/26] mkfs: enable online repair if all backrefs are enabled

Quoted 6 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If all backreferences are enabled in the filesystem, then enable online repair by default if the user didn't supply any other autofsck setting. Users might as well get full self-repair capability if they're paying for the extra metadata.

Does this cause scrub to run by default or just healer on demand? People might not be happy about the former.

#70 lore raw

Re: [PATCH 24/26] mkfs: enable online repair if all backrefs are enabled

Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If all backreferences are enabled in the filesystem, then enable online repair by default if the user didn't supply any other autofsck setting. Users might as well get full self-repair capability if they're paying for the extra metadata.

Does this cause scrub to run by default or just healer on demand? People might not be happy about the former.

Ultimately it's up to the distro to decide if (a) they turn on the kernel support and (b) enable the systemd services by default. Setting the fsproperty just means that you'll get different levels of online repair functionality if the user/sysadmin/crond actually invoke the services.

(That said, I was wondering if it was time to get rid of all the Kconfig options...)

--D

#71 lore raw

Re: [PATCH 24/26] mkfs: enable online repair if all backrefs are enabled

Quoted 12 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If all backreferences are enabled in the filesystem, then enable online repair by default if the user didn't supply any other autofsck setting. Users might as well get full self-repair capability if they're paying for the extra metadata.

Does this cause scrub to run by default or just healer on demand? People might not be happy about the former.

Quoted 8 lines

Ultimately it's up to the distro to decide if (a) they turn on the kernel support and (b) enable the systemd services by default. Setting the fsproperty just means that you'll get different levels of online repair functionality if the user/sysadmin/crond actually invoke the services.

(That said, I was wondering if it was time to get rid of all the Kconfig options...)

And here's where I'll add the note about distro policies:

"mkfs: enable online repair if all backrefs are enabled

"If all backreferences are enabled in the filesystem, then enable online repair by default if the user didn't supply any other autofsck setting. Users might as well get full self-repair capability if they're paying for the extra metadata.

"Note that it's up to each distro to enable the systemd services according to their own service activation policies. Debian policy is to enable all systemd services at package installation but they don't enable online fsck in their Kconfig so the services won't activate. RHEL and SUSE policy requires sysadmins to enable them explicitly unless the OS vendor also ships a systemd preset file enabling the services. Distros without systemd won't get any of the systemd services, obviously."

--D

#72 lore raw

PATCH 25/26 debian: enable xfs_healer on the root filesystem by default

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that we're finished building autonomous repair, enable the service on the root filesystem by default. The root filesystem is mounted by the initrd prior to starting systemd, which is why the udev rule cannot autostart the service for the root filesystem.

dh_installsystemd won't activate a template service (aka one with an at-sign in the name) even if it provides a DefaultInstance directive to make that possible. Use a fugly shim for this.

---
 debian/postinst |    8 ++++++++
 debian/prerm    |   13 +++++++++++++
 debian/rules    |    3 ++-
 3 files changed, 23 insertions(+), 1 deletion(-)
 create mode 100644 debian/prerm
debian/postinst +8 -0
index d11c8d94a3cbe4..966dbb7626cab3 100644
@@ -21,5 +21,13 @@ case "${1}" in
 esac
 
 #DEBHELPER#
+#
+# dh_installsystemd doesn't handle template services even if we supply a
+# default instance, so we'll install it here.
+if [ -z "${DPKG_ROOT:-}" ] && [ -d /run/systemd/system ] ; then
+	if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
+		/bin/systemctl enable [email protected] || true
+	fi
+fi
 
 exit 0
debian/prerm new file +13 -0
new file mode 100644
index 00000000000000..c526dcdd1d7103
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -e
+
+# dh_installsystemd doesn't handle template services even if we supply a
+# default instance, so we'll install it here.
+if [ -z "${DPKG_ROOT:-}" ] && [ "$1" = remove ] && [ -d /run/systemd/system ] ; then
+	/bin/systemctl disable [email protected] || true
+fi
+
+#DEBHELPER#
+
+exit 0
debian/rules +2 -1
index 7c9f90e6c483ff..aaf99a95ce3df5 100755
@@ -97,4 +97,5 @@ override_dh_installdocs:
 	dh_installdocs -XCHANGES
 
 override_dh_installsystemd:
-	dh_installsystemd -p xfsprogs --no-restart-after-upgrade --no-stop-on-upgrade system-xfs_scrub.slice xfs_scrub_all.timer
+	dh_installsystemd -p xfsprogs --no-restart-after-upgrade --no-stop-on-upgrade system-xfs_scrub.slice xfs_scrub_all.timer system-xfs_healer.slice
+	dh_installsystemd -p xfsprogs --restart-after-upgrade xfs_healer_start.service
#73 lore raw

Re: [PATCH 25/26] debian: enable xfs_healer on the root filesystem by default

Quoted 10 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that we're finished building autonomous repair, enable the service on the root filesystem by default. The root filesystem is mounted by the initrd prior to starting systemd, which is why the udev rule cannot autostart the service for the root filesystem.

dh_installsystemd won't activate a template service (aka one with an at-sign in the name) even if it provides a DefaultInstance directive to make that possible. Use a fugly shim for this.

Given that this is brand new code it feels a bit too early. But maybe that's just me.

#74 lore raw

Re: [PATCH 25/26] debian: enable xfs_healer on the root filesystem by default

Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that we're finished building autonomous repair, enable the service on the root filesystem by default. The root filesystem is mounted by the initrd prior to starting systemd, which is why the udev rule cannot autostart the service for the root filesystem.

Quoted 6 lines
Quoted 3 lines

dh_installsystemd won't activate a template service (aka one with an at-sign in the name) even if it provides a DefaultInstance directive to make that possible. Use a fugly shim for this.

Given that this is brand new code it feels a bit too early. But maybe that's just me.

A lot depends on the distro -- RHEL and SUSE require the sysadmin to activate services. Debian turns on any service shipping in a package by default, which is sort of funny since they don't enable online fsck in their kernel at all, so all the healer services fail the --supported checks and deactivate immediately.

(By contrast stock OL doesn't enable the service but enables scrub in the kernel; and UEK enables everything.)

--D

#75 lore raw

Re: [PATCH 25/26] debian: enable xfs_healer on the root filesystem by default

Quoted 5 lines · Darrick J. Wong

A lot depends on the distro -- RHEL and SUSE require the sysadmin to activate services. Debian turns on any service shipping in a package by default, which is sort of funny since they don't enable online fsck in their kernel at all, so all the healer services fail the --supported checks and deactivate immediately.

So this patch doesn't make much sense right now?

Either way it really should have these details in the commit log.

#76 lore raw

Re: [PATCH 25/26] debian: enable xfs_healer on the root filesystem by default

Quoted 10 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

A lot depends on the distro -- RHEL and SUSE require the sysadmin to activate services. Debian turns on any service shipping in a package by default, which is sort of funny since they don't enable online fsck in their kernel at all, so all the healer services fail the --supported checks and deactivate immediately.

So this patch doesn't make much sense right now?

Either way it really should have these details in the commit log.

<shrug> I'll amend the commit message:

Note that this won't do much right now because Debian doesn't enable online fsck in their kernels, so the ExecCondition will return false and the service won't actually activate.

--D

#77 lore raw

Re: [PATCH 25/26] debian: enable xfs_healer on the root filesystem by default

Quoted 10 lines · Darrick J. Wong
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

A lot depends on the distro -- RHEL and SUSE require the sysadmin to activate services. Debian turns on any service shipping in a package by default, which is sort of funny since they don't enable online fsck in their kernel at all, so all the healer services fail the --supported checks and deactivate immediately.

So this patch doesn't make much sense right now?

Quoted 7 lines
Quoted 1 line

Either way it really should have these details in the commit log.

<shrug> I'll amend the commit message:

Note that this won't do much right now because Debian doesn't enable online fsck in their kernels, so the ExecCondition will return false and the service won't actually activate.

Though now I see that the first part of the commit message is also out of date (we don't do udev anymore) so let's just replace the whole thing with:

"debian: enable xfs_healer on the root filesystem by default

"Now that we're finished building autonomous repair, enable the healer service on the root filesystem by default. The root filesystem is mounted by the initrd prior to starting systemd, which is why the xfs_healer_start service cannot autostart the service for the root filesystem.

"dh_installsystemd won't activate a template service (aka one with an at-sign in the name) even if it provides a DefaultInstance directive to make that possible. Hence we enable this explicitly via the postinst script.

"Note that Debian enables services by default upon package installation, so this is consistent with their policies. Their kernel doesn't enable online fsck, so healer won't do much more than monitor for corruptions and log them."

--D

#78 lore raw

PATCH 26/26 debian/control: listify the build dependencies

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

This will make it less gross to add more build deps later.

---
 debian/control |   14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
debian/control +13 -1
index 66b0a47a36ee24..7837019804e93a 100644
@@ -3,7 +3,19 @@ Section: admin
 Priority: optional
 Maintainer: XFS Development Team <[email protected]>
 Uploaders: Nathan Scott <[email protected]>, Anibal Monsalve Salazar <[email protected]>, Bastian Germann <[email protected]>
-Build-Depends: libinih-dev (>= 53), uuid-dev, debhelper (>= 12), gettext, libtool, libedit-dev, libblkid-dev (>= 2.17), linux-libc-dev, libdevmapper-dev, libicu-dev, pkg-config, liburcu-dev, systemd-dev | systemd (<< 253-2~)
+Build-Depends: debhelper (>= 12),
+ gettext,
+ libblkid-dev (>= 2.17),
+ libdevmapper-dev,
+ libedit-dev,
+ libicu-dev,
+ libinih-dev (>= 53),
+ libtool,
+ liburcu-dev,
+ linux-libc-dev,
+ pkg-config,
+ systemd-dev | systemd (<< 253-2~),
+ uuid-dev
 Standards-Version: 4.0.0
 Homepage: https://xfs.wiki.kernel.org/
 
#79 lore raw

Re: [PATCH 26/26] debian/control: listify the build dependencies

Quoted 3 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

This will make it less gross to add more build deps later.

Looks good, but should this go to the beginning of the series?

#80 lore raw

Re: [PATCH 26/26] debian/control: listify the build dependencies

Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

This will make it less gross to add more build deps later.

Looks good, but should this go to the beginning of the series?

Fine with me.

--D

#81 lore raw

PATCH v8 1/2 fstests: test generic file IO error reporting

In reply to Darrick J. Wong

Hi all,

Refactor the iomap file I/O error handling code so that failures are reported in a generic way to fsnotify. Then connect the XFS health reporting to the same fsnotify, and now XFS can notify userspace of problems.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=filesystem-error-reporting

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=filesystem-error-reporting
---
Commits in this patchset:
* generic: test fsnotify filesystem error reporting

---
 src/Makefile           |    2 
 src/fs-monitor.c       |  155 +++++++++++++++++++++++++++++++++
 tests/generic/1838     |  228 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/1838.out |   20 ++++
 4 files changed, 404 insertions(+), 1 deletion(-)
 create mode 100644 src/fs-monitor.c
 create mode 100755 tests/generic/1838
 create mode 100644 tests/generic/1838.out
#82 lore raw

PATCH 1/1 generic: test fsnotify filesystem error reporting

From: Darrick J. Wong <[email protected]>

Test the fsnotify filesystem error reporting.

---
 src/Makefile           |    2 
 src/fs-monitor.c       |  155 +++++++++++++++++++++++++++++++++
 tests/generic/1838     |  228 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/1838.out |   20 ++++
 4 files changed, 404 insertions(+), 1 deletion(-)
 create mode 100644 src/fs-monitor.c
 create mode 100755 tests/generic/1838
 create mode 100644 tests/generic/1838.out
src/Makefile +1 -1
index 577d816ae859b6..1c761da0ccff20 100644
@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 	fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \
 	detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \
 	uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \
-	rw_hint
+	rw_hint fs-monitor
 
 EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \
 	      btrfs_crc32c_forged_name.py popdir.pl popattr.py \
src/fs-monitor.c new file +155 -0

155 lines not on this page · raw message

tests/generic/1838 new file +228 -0

228 lines not on this page · raw message

tests/generic/1838.out new file +20 -0
new file mode 100644
index 00000000000000..adae590fe0b2ea
@@ -0,0 +1,20 @@
+QA output created by 1838
+Try buffered read
+pread: Input/output error
+Try directio read
+pread: Input/output error
+Try directio write
+pwrite: Input/output error
+Try buffered write
+fsync: Input/output error
+Fix device
+Kill fsnotify
+	Generic Error Record: len=XXX
+	error: 5
+FAN_FS_ERROR (len=XXX)
+Remount and restart fsnotify
+Try buffered read again
+Try directio read again
+Try directio write again
+Try buffered write again
+Kill fsnotify again
#83 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 4 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Test the fsnotify filesystem error reporting.

For the record, I feel that I need to say to all the people whom we pushed back on fanotify tests in fstests until there was a good enough reason to do so, that this seems like a good reason to do so ;)

But also for future test writers, please note that FAN_FS_ERROR is an exception to the rule and please keep writing new fanotify/inotify tests in LTP (until there is a good enough reason...)

Quoted 13 lines
---
 src/Makefile           |    2
 src/fs-monitor.c       |  155 +++++++++++++++++++++++++++++++++
 tests/generic/1838     |  228 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/1838.out |   20 ++++
 4 files changed, 404 insertions(+), 1 deletion(-)
 create mode 100644 src/fs-monitor.c
 create mode 100755 tests/generic/1838
 create mode 100644 tests/generic/1838.out

...

Quoted 40 lines
tests/generic/1838 new file +34 -0
new file mode 100755
index 00000000000000..087851ddcbdb44
@@ -0,0 +1,228 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1838
+#
+# Check that fsnotify can report file IO errors.
+
+. ./common/preamble
+_begin_fstest auto quick eio selfhealing
+
+# Override the default cleanup function.
+_cleanup()
+{
+       cd /
+       test -n "$fsmonitor_pid" && kill -TERM $fsmonitor_pid
+       rm -f $tmp.*
+       _dmerror_cleanup
+}
+
+# Import common functions.
+. ./common/fuzzy
+. ./common/filter
+. ./common/dmerror
+. ./common/systemd
+
+case "$FSTYP" in
+xfs)
+       # added as a part of xfs health monitoring
+       _require_xfs_io_command healthmon
+       # no out of place writes
+       _require_no_xfs_always_cow
+       ;;
+ext4)

tests/generic/1838 line 44

Quoted 10 lines
tests/generic/1838 new file +10 -0
+       # added at the same time as uevents
+       modprobe fs-$FSTYP
+       test -e /sys/fs/ext4/features/uevents || \
+               _notrun "$FSTYP does not support fsnotify ioerrors"
+       ;;
+*)
+       _notrun "$FSTYP does not support fsnotify ioerrors"
+       ;;
+esac
+

_require_fsnotify_errors ?

Quoted 7 lines
tests/generic/1838 new file +7 -0
+_require_scratch
+_require_dm_target error
+_require_test_program fs-monitor
+_require_xfs_io_command "fiemap"
+_require_odirect
+
+# fsnotify only gives us a file handle, the error number, and the number of

tests/generic/1838 line 61

Quoted 10 lines
tests/generic/1838 new file +10 -0
+# times it was seen in between event deliveries.   The handle is mostly useless
+# since we have no generic way to map that to a file path.  Therefore we can
+# only coalesce all the I/O errors into one report.
+filter_fsnotify_errors() {
+       _filter_scratch | \
+               grep -E '(FAN_FS_ERROR|Generic Error Record|error: 5)' | \
+               sed -e "s/len=[0-9]*/len=XXX/g" | \
+               sort | \
+               uniq
+}

move to common/filter?

Apart from those nits, no further comments.

Thanks,
Amir.

#84 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 9 lines · Amir Goldstein
Quoted 4 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Test the fsnotify filesystem error reporting.

For the record, I feel that I need to say to all the people whom we pushed back on fanotify tests in fstests until there was a good enough reason to do so, that this seems like a good reason to do so ;)

Who pushed backed on that? Because IMHO hiding stuff in ltp is a sure way it doesn't get exercisesd regularly?

#85 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Amir Goldstein
Quoted 3 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Quoted 9 lines
Quoted 5 lines
Quoted 1 line

Test the fsnotify filesystem error reporting.

For the record, I feel that I need to say to all the people whom we pushed back on fanotify tests in fstests until there was a good enough reason to do so, that this seems like a good reason to do so ;)

Who pushed backed on that? Because IMHO hiding stuff in ltp is a sure way it doesn't get exercisesd regularly?

Jan and myself pushed back on adding generic fanotify tests to fstest because we already have most fanotify tests in LTP.

LTP is run by many testers on many boxes and many release kernels and we are happy with this project to host tests for the subsystem that we maintain.

Thanks,
Amir.

reviewer
#86 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

In reply to Christoph Hellwig
Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · Amir Goldstein
Quoted 3 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Test the fsnotify filesystem error reporting.

For the record, I feel that I need to say to all the people whom we pushed back on fanotify tests in fstests until there was a good enough reason to do so, that this seems like a good reason to do so ;)

Who pushed backed on that? Because IMHO hiding stuff in ltp is a sure way it doesn't get exercisesd regularly?

Amir wrote it well, I'd just add the 0-day runs LTP, distro people run LTP and lot of other test bots also run LTP so I wouldn't say fsnotify tests are not exercised regularly. For record I don't expect regular filesystem developers to need to run fsnotify tests as the code is generally well separated from individual filesystems. Filesystem error reporting is kind of special in this regard so I agree having it in fstests makes sense.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#87 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

In reply to Darrick J. Wong

src/fs-monitor.c line 4

Quoted 4 lines
src/fs-monitor.c new file +4 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2021, Collabora Ltd.
+ */

Where is this coming from?

src/fs-monitor.c line 16

Quoted 3 lines
src/fs-monitor.c new file +3 -0
+#ifndef __GLIBC__
+#include <asm-generic/int-ll64.h>
+#endif

And what is this for? Looks pretty whacky.

Quoted 7 lines
tests/generic/1838 new file +7 -0
+case "$FSTYP" in
+xfs)
+	# added as a part of xfs health monitoring
+	_require_xfs_io_command healthmon
+	# no out of place writes
+	_require_no_xfs_always_cow
+	;;

tests/generic/1838 line 43

Quoted 10 lines
tests/generic/1838 new file +10 -0
+ext4)
+	# added at the same time as uevents
+	modprobe fs-$FSTYP
+	test -e /sys/fs/ext4/features/uevents || \
+		_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+*)
+	_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+esac

Please abstract this out into a documented helper in common/

tests/generic/1838 line 75

Quoted 11 lines
tests/generic/1838 new file +11 -0
+#
+# The dm-error map added by this test doesn't work on zoned devices because
+# table sizes need to be aligned to the zone size, and even for zoned on
+# conventional this test will get confused because of the internal RT device.
+#
+# That check requires a mounted file system, so do a dummy mount before setting
+# up DM.
+#
+_scratch_mount
+test $FSTYP = xfs && _require_xfs_scratch_non_zoned
+_scratch_unmount

Hmm, this is a bit sad. Can we align the map? Or should we carve in and add proper error injection to the block code, which has been somewhere on my todo list forever because dm-error and friends are so painful to setup. Maybe I need to expedite that.

#88 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 6 lines · Christoph Hellwig

src/fs-monitor.c line 4

Quoted 4 lines
src/fs-monitor.c new file +4 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2021, Collabora Ltd.
+ */

Where is this coming from?

This code is heavily based, if not the same, to what I originally wrote as a kernel tree "samples/fs-monitor.c" when I was employed by Collabora. I appreciate Darrick keeping the note actually.

Quoted 6 lines

src/fs-monitor.c line 16

Quoted 4 lines
src/fs-monitor.c new file +3 -0
+#ifndef __GLIBC__
+#include <asm-generic/int-ll64.h>
+#endif

And what is this for? Looks pretty whacky.

Comes from kernel commit 3193e8942fc7 ("samples: fix building fs-monitor on musl systems") to fix building with musl. We don't need it here.

Signature · 1 line
Gabriel Krisman Bertazi
#89 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 9 lines · Gabriel Krisman Bertazi
Quoted 6 lines · Christoph Hellwig

src/fs-monitor.c line 4

Quoted 4 lines
src/fs-monitor.c new file +4 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2021, Collabora Ltd.
+ */

Where is this coming from?

Quoted 3 lines

This code is heavily based, if not the same, to what I originally wrote as a kernel tree "samples/fs-monitor.c" when I was employed by Collabora. I appreciate Darrick keeping the note actually.

The note is good. But if we import code from somewhere, we should document where it is coming from, both for attribution and to ease any future resyncs if needed.

Quoted 8 lines
Quoted 5 lines

src/fs-monitor.c line 16

Quoted 3 lines
src/fs-monitor.c new file +3 -0
+#ifndef __GLIBC__
+#include <asm-generic/int-ll64.h>
+#endif

And what is this for? Looks pretty whacky.

Comes from kernel commit 3193e8942fc7 ("samples: fix building fs-monitor on musl systems") to fix building with musl. We don't need it here.

In the place that needs it it really should have a comment explainig the logic behind it.

#90 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Gabriel Krisman Bertazi
Quoted 6 lines · Christoph Hellwig

src/fs-monitor.c line 4

Quoted 4 lines
src/fs-monitor.c new file +4 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2021, Collabora Ltd.
+ */

Where is this coming from?

Quoted 7 lines
Quoted 3 lines

This code is heavily based, if not the same, to what I originally wrote as a kernel tree "samples/fs-monitor.c" when I was employed by Collabora. I appreciate Darrick keeping the note actually.

The note is good. But if we import code from somewhere, we should document where it is coming from, both for attribution and to ease any future resyncs if needed.

Yeah, I copied this straight from the kernel tree, which is why it contains this wart:

Quoted 11 lines
Quoted 8 lines
Quoted 5 lines

src/fs-monitor.c line 16

Quoted 3 lines
src/fs-monitor.c new file +3 -0
+#ifndef __GLIBC__
+#include <asm-generic/int-ll64.h>
+#endif

And what is this for? Looks pretty whacky.

Comes from kernel commit 3193e8942fc7 ("samples: fix building fs-monitor on musl systems") to fix building with musl. We don't need it here.

In the place that needs it it really should have a comment explainig the logic behind it.

I don't know that people *don't* try to run fstests with musl. But as they seem surprisingly patient with continuously fixing up xfsprogs, perhaps it's ok to clean this up on the way into fstests.

I'll add more attribution for the c file pointing back to where it came from.

--D

#91 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

In reply to Christoph Hellwig
Quoted 24 lines · Christoph Hellwig

src/fs-monitor.c line 4

Quoted 4 lines
src/fs-monitor.c new file +4 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2021, Collabora Ltd.
+ */

Where is this coming from?

src/fs-monitor.c line 16

Quoted 3 lines
src/fs-monitor.c new file +3 -0
+#ifndef __GLIBC__
+#include <asm-generic/int-ll64.h>
+#endif

And what is this for? Looks pretty whacky.

Quoted 11 lines
tests/generic/1838 new file +11 -0
+case "$FSTYP" in
+xfs)
+	# added as a part of xfs health monitoring
+	_require_xfs_io_command healthmon
+	# no out of place writes
+	_require_no_xfs_always_cow
+	;;
+ext4)
+	# added at the same time as uevents
+	modprobe fs-$FSTYP
+	test -e /sys/fs/ext4/features/uevents || \
Quoted 8 lines

tests/generic/1838 line 43

Quoted 6 lines
tests/generic/1838 new file +6 -0
+		_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+*)
+	_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+esac

Please abstract this out into a documented helper in common/

Ok. I'm not sure how to check for feature support on ext4 anymore since the uevents patch didn't get merged, and then I clearly forgot to rip that out of this helper here.

Quoted 8 lines
Quoted 8 lines
tests/generic/1838 new file +8 -0
+#
+# The dm-error map added by this test doesn't work on zoned devices because
+# table sizes need to be aligned to the zone size, and even for zoned on
+# conventional this test will get confused because of the internal RT device.
+#
+# That check requires a mounted file system, so do a dummy mount before setting
+# up DM.
+#
Quoted 8 lines

tests/generic/1838 line 75

Quoted 3 lines
tests/generic/1838 new file +3 -0
+_scratch_mount
+test $FSTYP = xfs && _require_xfs_scratch_non_zoned
+_scratch_unmount

Hmm, this is a bit sad. Can we align the map? Or should we carve in and add proper error injection to the block code, which has been somewhere on my todo list forever because dm-error and friends are so painful to setup. Maybe I need to expedite that.

I think it's theoretically possible to figure out that there's a zone size and then round outwards the error-target part of the dm table to align with a zone. I have a lot more doubts about whether or not doing that in bash/awk is a good idea though. It'd be a lot easier if either the block layer did error injection or if someone just fixes those limitations in dm itself.

--D

#92 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 13 lines · Darrick J. Wong
Quoted 4 lines
Quoted 4 lines
tests/generic/1838 new file +4 -0
+ext4)
+	# added at the same time as uevents
+	modprobe fs-$FSTYP
+	test -e /sys/fs/ext4/features/uevents || \
Quoted 8 lines

tests/generic/1838 line 43

Quoted 6 lines
tests/generic/1838 new file +6 -0
+		_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+*)
+	_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+esac

Please abstract this out into a documented helper in common/

Quoted 3 lines

Ok. I'm not sure how to check for feature support on ext4 anymore since the uevents patch didn't get merged, and then I clearly forgot to rip that out of this helper here.

Oh. Well, drop that then and move the xfs side and the default n into a common helper instead of hardcoding it in the test.

Quoted 7 lines
Quoted 3 lines

and add proper error injection to the block code, which has been somewhere on my todo list forever because dm-error and friends are so painful to setup. Maybe I need to expedite that.

I think it's theoretically possible to figure out that there's a zone size and then round outwards the error-target part of the dm table to align with a zone.

It's the sysfs chunk size. btrfs/237 harcodes reading that out, which could be easily lifted into a helper.

Quoted 4 lines

I have a lot more doubts about whether or not doing
that in bash/awk is a good idea though. It'd be a lot easier if either the block layer did error injection or if someone just fixes those limitations in dm itself.

I'll sign up to do the block layer stuff. Doing so should allow us to run a lot more of the error injetion tests on zoned xfs, which would be good. So I guess you should keep it as-is for now, and I'll do a sweep later.

#93 lore raw

Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting

Quoted 14 lines · Christoph Hellwig
Quoted 13 lines · Darrick J. Wong
Quoted 12 lines

tests/generic/1838 line 43

Quoted 10 lines
tests/generic/1838 new file +10 -0
+ext4)
+	# added at the same time as uevents
+	modprobe fs-$FSTYP
+	test -e /sys/fs/ext4/features/uevents || \
+		_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+*)
+	_notrun "$FSTYP does not support fsnotify ioerrors"
+	;;
+esac

Please abstract this out into a documented helper in common/

Quoted 6 lines
Quoted 3 lines

Ok. I'm not sure how to check for feature support on ext4 anymore since the uevents patch didn't get merged, and then I clearly forgot to rip that out of this helper here.

Oh. Well, drop that then and move the xfs side and the default n into a common helper instead of hardcoding it in the test.

/me discovers that Baolin Liu added a "err_report_sec" sysfs knob to ext4 in 7.0-rc1, so I can just change the helper to look for that. I'll move the logic to common/rc.

Quoted 16 lines
Quoted 7 lines
Quoted 3 lines

and add proper error injection to the block code, which has been somewhere on my todo list forever because dm-error and friends are so painful to setup. Maybe I need to expedite that.

I think it's theoretically possible to figure out that there's a zone size and then round outwards the error-target part of the dm table to align with a zone.

It's the sysfs chunk size. btrfs/237 harcodes reading that out, which could be easily lifted into a helper.

Quoted 4 lines

I have a lot more doubts about whether or not doing
that in bash/awk is a good idea though. It'd be a lot easier if either the block layer did error injection or if someone just fixes those limitations in dm itself.

Quoted 4 lines

I'll sign up to do the block layer stuff. Doing so should allow us to run a lot more of the error injetion tests on zoned xfs, which would be good. So I guess you should keep it as-is for now, and I'll do a sweep later.

Ok, thanks. :)

--D

#94 lore raw

PATCH v8 2/2 fstests: autonomous self healing of filesystems

In reply to Darrick J. Wong

Hi all,

This series adds functionality and regression tests for the automated self healing daemon for xfs.

v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=health-monitoring

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=health-monitoring

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=health-monitoring
---
Commits in this patchset:
* xfs: test health monitoring code
* xfs: test for metadata corruption error reporting via healthmon
* xfs: test io error reporting via healthmon
* xfs: set up common code for testing xfs_healer
* xfs: test xfs_healer's event handling
* xfs: test xfs_healer can fix a filesystem
* xfs: test xfs_healer can report file I/O errors
* xfs: test xfs_healer can report file media errors
* xfs: test xfs_healer can report filesystem shutdowns
* xfs: test xfs_healer can initiate full filesystem repairs
* xfs: test xfs_healer can follow mount moves
* xfs: test xfs_healer wont repair the wrong filesystem
* xfs: test xfs_healer background service

---
 common/config       |   14 +++
 common/rc           |   15 ++++
 common/systemd      |   32 ++++++++
 common/xfs          |  114 ++++++++++++++++++++++++++++
 doc/group-names.txt |    1 
 tests/xfs/1878      |   93 +++++++++++++++++++++++
 tests/xfs/1878.out  |   10 ++
 tests/xfs/1879      |   93 +++++++++++++++++++++++
 tests/xfs/1879.out  |    8 ++
 tests/xfs/1882      |   44 +++++++++++
 tests/xfs/1882.out  |    2 
 tests/xfs/1884      |   89 ++++++++++++++++++++++
 tests/xfs/1884.out  |    2 
 tests/xfs/1885      |   53 +++++++++++++
 tests/xfs/1885.out  |    5 +
 tests/xfs/1896      |  210 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1896.out  |   21 +++++
 tests/xfs/1897      |  172 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1897.out  |    7 ++
 tests/xfs/1898      |   37 +++++++++
 tests/xfs/1898.out  |    4 +
 tests/xfs/1899      |  108 ++++++++++++++++++++++++++
 tests/xfs/1899.out  |    3 +
 tests/xfs/1900      |  115 ++++++++++++++++++++++++++++
 tests/xfs/1900.out  |    2 
 tests/xfs/1901      |  137 +++++++++++++++++++++++++++++++++
 tests/xfs/1901.out  |    2 
 tests/xfs/1902      |  152 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1902.out  |    2 
 tests/xfs/802       |    4 -
 30 files changed, 1549 insertions(+), 2 deletions(-)
 create mode 100755 tests/xfs/1878
 create mode 100644 tests/xfs/1878.out
 create mode 100755 tests/xfs/1879
 create mode 100644 tests/xfs/1879.out
 create mode 100755 tests/xfs/1882
 create mode 100644 tests/xfs/1882.out
 create mode 100755 tests/xfs/1884
 create mode 100644 tests/xfs/1884.out
 create mode 100755 tests/xfs/1885
 create mode 100644 tests/xfs/1885.out
 create mode 100755 tests/xfs/1896
 create mode 100644 tests/xfs/1896.out
 create mode 100755 tests/xfs/1897
 create mode 100755 tests/xfs/1897.out
 create mode 100755 tests/xfs/1898
 create mode 100755 tests/xfs/1898.out
 create mode 100755 tests/xfs/1899
 create mode 100644 tests/xfs/1899.out
 create mode 100755 tests/xfs/1900
 create mode 100755 tests/xfs/1900.out
 create mode 100755 tests/xfs/1901
 create mode 100755 tests/xfs/1901.out
 create mode 100755 tests/xfs/1902
 create mode 100755 tests/xfs/1902.out
#95 lore raw

PATCH 1/13 xfs: test health monitoring code

From: Darrick J. Wong <[email protected]>

Add some functionality tests for the new health monitoring code.

---
 doc/group-names.txt |    1 +
 tests/xfs/1885      |   53 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1885.out  |    5 +++++
 3 files changed, 59 insertions(+)
 create mode 100755 tests/xfs/1885
 create mode 100644 tests/xfs/1885.out
doc/group-names.txt +1 -0
index 10b49e50517797..158f84d36d3154 100644
@@ -117,6 +117,7 @@ samefs			overlayfs when all layers are on the same fs
 scrub			filesystem metadata scrubbers
 seed			btrfs seeded filesystems
 seek			llseek functionality
+selfhealing		self healing filesystem code
 selftest		tests with fixed results, used to validate testing setup
 send			btrfs send/receive
 shrinkfs		decreasing the size of a filesystem
tests/xfs/1885 new file +53 -0

53 lines not on this page · raw message

tests/xfs/1885.out new file +5 -0
new file mode 100644
index 00000000000000..f152cef0525609
@@ -0,0 +1,5 @@
+QA output created by 1885
+mount refcount is in range
+mount + healthmon refcount is in range
+healthmon refcount is in range
+end refcount is in range
#96 lore raw

Re: [PATCH 01/13] xfs: test health monitoring code

Quoted 32 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some functionality tests for the new health monitoring code.

---
 doc/group-names.txt |    1 +
 tests/xfs/1885      |   53 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1885.out  |    5 +++++
 3 files changed, 59 insertions(+)
 create mode 100755 tests/xfs/1885
 create mode 100644 tests/xfs/1885.out
doc/group-names.txt +1 -0
index 10b49e50517797..158f84d36d3154 100644
@@ -117,6 +117,7 @@ samefs			overlayfs when all layers are on the same fs
 scrub			filesystem metadata scrubbers
 seed			btrfs seeded filesystems
 seek			llseek functionality
+selfhealing		self healing filesystem code
 selftest		tests with fixed results, used to validate testing setup
 send			btrfs send/receive
 shrinkfs		decreasing the size of a filesystem
tests/xfs/1885 new file +0 -0
new file mode 100755
index 00000000000000..1d75ef19c7c9d9

tests/xfs/1885 line 10

Quoted 10 lines
tests/xfs/1885 new file +10 -0
@@ -0,0 +1,53 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1885
+#
+# Make sure that healthmon handles module refcount correctly.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing

I found this test is quick enough, how about add it into "quick" group.

tests/xfs/1885 line 13

Quoted 3 lines
tests/xfs/1885 new file +3 -0
+
+. ./common/filter
+. ./common/module

Which helper is this "module" file being included for?

tests/xfs/1885 line 16

Quoted 3 lines
tests/xfs/1885 new file +3 -0
+
+refcount_file="/sys/module/xfs/refcnt"
+test -e "$refcount_file" || _notrun "cannot find xfs module refcount"

Or did you intend to add this part as a helper into common/module?

Quoted 18 lines
tests/xfs/1885 new file +18 -0
+
+_require_test
+_require_xfs_io_command healthmon
+
+# Capture mod refcount without the test fs mounted
+_test_unmount
+init_refcount="$(cat "$refcount_file")"
+
+# Capture mod refcount with the test fs mounted
+_test_mount
+nomon_mount_refcount="$(cat "$refcount_file")"
+
+# Capture mod refcount with test fs mounted and the healthmon fd open.
+# Pause the xfs_io process so that it doesn't actually respond to events.
+$XFS_IO_PROG -c 'healthmon -c -v' $TEST_DIR >> $seqres.full &
+sleep 0.5
+kill -STOP %1
+mon_mount_refcount="$(cat "$refcount_file")"

tests/xfs/1885 line 44

Quoted 10 lines
tests/xfs/1885 new file +10 -0
+
+# Capture mod refcount with only the healthmon fd open.
+_test_unmount
+mon_nomount_refcount="$(cat "$refcount_file")"
+
+# Capture mod refcount after continuing healthmon (which should exit due to the
+# unmount) and killing it.
+kill -CONT %1
+kill %1
+wait

We typically ensure that background processes are handled within the _cleanup function.

tests/xfs/1885 line 53

Quoted 9 lines
tests/xfs/1885 new file +9 -0
+nomon_nomount_refcount="$(cat "$refcount_file")"
+
+_within_tolerance "mount refcount" "$nomon_mount_refcount" "$((init_refcount + 1))" 0 -v
+_within_tolerance "mount + healthmon refcount" "$mon_mount_refcount" "$((init_refcount + 2))" 0 -v
+_within_tolerance "healthmon refcount" "$mon_nomount_refcount" "$((init_refcount + 1))" 0 -v
+_within_tolerance "end refcount" "$nomon_nomount_refcount" "$init_refcount" 0 -v
+
+status=0
+exit

_exit 0

Quoted 12 lines
tests/xfs/1885.out new file +5 -0
new file mode 100644
index 00000000000000..f152cef0525609
@@ -0,0 +1,5 @@
+QA output created by 1885
+mount refcount is in range
+mount + healthmon refcount is in range
+healthmon refcount is in range
+end refcount is in range
#97 lore raw

Re: [PATCH 01/13] xfs: test health monitoring code

Quoted 37 lines · Zorro Lang
Quoted 36 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some functionality tests for the new health monitoring code.

---
 doc/group-names.txt |    1 +
 tests/xfs/1885      |   53 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1885.out  |    5 +++++
 3 files changed, 59 insertions(+)
 create mode 100755 tests/xfs/1885
 create mode 100644 tests/xfs/1885.out
doc/group-names.txt +1 -0
index 10b49e50517797..158f84d36d3154 100644
@@ -117,6 +117,7 @@ samefs			overlayfs when all layers are on the same fs
 scrub			filesystem metadata scrubbers
 seed			btrfs seeded filesystems
 seek			llseek functionality
+selfhealing		self healing filesystem code
 selftest		tests with fixed results, used to validate testing setup
 send			btrfs send/receive
 shrinkfs		decreasing the size of a filesystem
tests/xfs/1885 new file +4 -0
new file mode 100755
index 00000000000000..1d75ef19c7c9d9
@@ -0,0 +1,53 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
Quoted 8 lines

tests/xfs/1885 line 10

Quoted 6 lines
tests/xfs/1885 new file +6 -0
+# FS QA Test 1885
+#
+# Make sure that healthmon handles module refcount correctly.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing

I found this test is quick enough, how about add it into "quick" group.

OK.

Quoted 5 lines

tests/xfs/1885 line 13

Quoted 3 lines
tests/xfs/1885 new file +3 -0
+
+. ./common/filter
+. ./common/module

Which helper is this "module" file being included for?

I think at one point I would rmmod/modprobe the module to force the refcount leak issue, but discovered there's a sysfs knob for that...

Quoted 5 lines

tests/xfs/1885 line 16

Quoted 3 lines
tests/xfs/1885 new file +3 -0
+
+refcount_file="/sys/module/xfs/refcnt"
+test -e "$refcount_file" || _notrun "cannot find xfs module refcount"

Or did you intend to add this part as a helper into common/module?

...so this probably should get refactored into a new helper.

Quoted 22 lines
Quoted 22 lines
tests/xfs/1885 new file +22 -0
+
+_require_test
+_require_xfs_io_command healthmon
+
+# Capture mod refcount without the test fs mounted
+_test_unmount
+init_refcount="$(cat "$refcount_file")"
+
+# Capture mod refcount with the test fs mounted
+_test_mount
+nomon_mount_refcount="$(cat "$refcount_file")"
+
+# Capture mod refcount with test fs mounted and the healthmon fd open.
+# Pause the xfs_io process so that it doesn't actually respond to events.
+$XFS_IO_PROG -c 'healthmon -c -v' $TEST_DIR >> $seqres.full &
+sleep 0.5
+kill -STOP %1
+mon_mount_refcount="$(cat "$refcount_file")"
+
+# Capture mod refcount with only the healthmon fd open.
+_test_unmount
+mon_nomount_refcount="$(cat "$refcount_file")"
Quoted 8 lines

tests/xfs/1885 line 44

Quoted 6 lines
tests/xfs/1885 new file +6 -0
+
+# Capture mod refcount after continuing healthmon (which should exit due to the
+# unmount) and killing it.
+kill -CONT %1
+kill %1
+wait

We typically ensure that background processes are handled within the _cleanup function.

oops, will clean that up.

$XFS_IO_PROG -c 'healthmon -c -v' $TEST_DIR >> $seqres.full & healer_pid=$!
...
kill $healer_pid

etc. Thanks for pointing that out.

--D

Quoted 25 lines

tests/xfs/1885 line 53

Quoted 9 lines
tests/xfs/1885 new file +9 -0
+nomon_nomount_refcount="$(cat "$refcount_file")"
+
+_within_tolerance "mount refcount" "$nomon_mount_refcount" "$((init_refcount + 1))" 0 -v
+_within_tolerance "mount + healthmon refcount" "$mon_mount_refcount" "$((init_refcount + 2))" 0 -v
+_within_tolerance "healthmon refcount" "$mon_nomount_refcount" "$((init_refcount + 1))" 0 -v
+_within_tolerance "end refcount" "$nomon_nomount_refcount" "$init_refcount" 0 -v
+
+status=0
+exit

_exit 0

Quoted 12 lines
tests/xfs/1885.out new file +5 -0
new file mode 100644
index 00000000000000..f152cef0525609
@@ -0,0 +1,5 @@
+QA output created by 1885
+mount refcount is in range
+mount + healthmon refcount is in range
+healthmon refcount is in range
+end refcount is in range
#98 lore raw

PATCH 2/13 xfs: test for metadata corruption error reporting via healthmon

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Check if we can detect runtime metadata corruptions via the health monitor.

---
 common/rc          |   10 ++++++
 tests/xfs/1879     |   93 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1879.out |    8 ++++
 3 files changed, 111 insertions(+)
 create mode 100755 tests/xfs/1879
 create mode 100644 tests/xfs/1879.out
common/rc +10 -0
index fd4ca9641822cf..38d4b500b3b51f 100644
@@ -3013,6 +3013,16 @@ _require_xfs_io_command()
 		echo $testio | grep -q "Inappropriate ioctl" && \
 			_notrun "xfs_io $command support is missing"
 		;;
+	"healthmon")
+		testio=`$XFS_IO_PROG -c "$command -p $param" $TEST_DIR 2>&1`
+		echo $testio | grep -q "bad argument count" && \
+			_notrun "xfs_io $command $param support is missing"
+		echo $testio | grep -q "Inappropriate ioctl" && \
+			_notrun "xfs_io $command $param ioctl support is missing"
+		echo $testio | grep -q "Operation not supported" && \
+			_notrun "xfs_io $command $param kernel support is missing"
+		param_checked="$param"
+		;;
 	"label")
 		testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1`
 		;;
tests/xfs/1879 new file +93 -0

93 lines not on this page · raw message

tests/xfs/1879.out new file +8 -0
new file mode 100644
index 00000000000000..2f6acbe1c4fb22
@@ -0,0 +1,8 @@
+QA output created by 1879
+Format and mount
+Runtime corruption detection
+VICTIM directory: sick
+Scrub corruption detection
+VICTIM directory: corrupt
+VICTIM directory: sick
+VICTIM parent: corrupt
#99 lore raw

PATCH 3/13 xfs: test io error reporting via healthmon

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a new test to make sure the kernel can report IO errors via health monitoring.

---
 tests/xfs/1878     |   93 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1878.out |   10 ++++++
 2 files changed, 103 insertions(+)
 create mode 100755 tests/xfs/1878
 create mode 100644 tests/xfs/1878.out
tests/xfs/1878 new file +93 -0
new file mode 100755
index 00000000000000..1ff6ae040fb193
@@ -0,0 +1,93 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1878
+#
+# Attempt to read and write a file in buffered and directio mode with the
+# health monitoring program running.  Check that healthmon observes all four
+# types of IO errors.
+#
+. ./common/preamble
+_begin_fstest auto quick eio selfhealing
+
+_cleanup()
+{
+	cd /
+	rm -rf $tmp.* $testdir
+	_dmerror_cleanup
+}
+
+. ./common/filter
+. ./common/dmerror
+
+_require_scratch_nocheck
+_require_xfs_io_command healthmon
+_require_dm_target error
+
+filter_healer_errors() {
+	_filter_scratch | \
+		grep -E '(buffered|directio)' | \
+		sed \
+		    -e 's/ino [0-9]*/ino NUM/g' \
+		    -e 's/gen 0x[0-9a-f]*/gen NUM/g' \
+		    -e 's/pos [0-9]*/pos NUM/g' \
+		    -e 's/len [0-9]*/len NUM/g' \
+		    -e 's|SCRATCH_MNT/a|VICTIM|g' \
+		    -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' | \
+		uniq
+}
+
+# Disable the scratch rt device to avoid test failures relating to the rt
+# bitmap consuming all the free space in our small data device.
+unset SCRATCH_RTDEV
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_dmerror_init no_log
+_dmerror_mount
+
+_require_fs_space $SCRATCH_MNT 65536
+
+# Create a file with written regions far enough apart that the pagecache can't
+# possibly be caching the regions with a single folio.
+testfile=$SCRATCH_MNT/fsync-err-test
+$XFS_IO_PROG -f \
+	-c 'pwrite -b 1m 0 1m' \
+	-c 'pwrite -b 1m 10g 1m' \
+	-c 'pwrite -b 1m 20g 1m' \
+	-c fsync $testfile >> $seqres.full
+
+# First we check if directio errors get reported
+$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT >> $tmp.healthmon &
+sleep 1	# wait for program to start up
+_dmerror_load_error_table
+$XFS_IO_PROG -d -c 'pwrite -b 256k 12k 16k' $testfile >> $seqres.full
+$XFS_IO_PROG -d -c 'pread -b 256k 10g 16k' $testfile >> $seqres.full
+_dmerror_load_working_table
+
+_dmerror_unmount
+wait	# for healthmon to finish
+_dmerror_mount
+
+# Next we check if buffered io errors get reported.  We have to write something
+# before loading the error table to ensure the dquots get loaded.
+$XFS_IO_PROG -c 'pwrite -b 256k 20g 1k' -c fsync $testfile >> $seqres.full
+$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT >> $tmp.healthmon &
+sleep 1	# wait for program to start up
+_dmerror_load_error_table
+$XFS_IO_PROG -c 'pread -b 256k 12k 16k' $testfile >> $seqres.full
+$XFS_IO_PROG -c 'pwrite -b 256k 20g 16k' -c fsync $testfile >> $seqres.full
+_dmerror_load_working_table
+
+_dmerror_unmount
+wait	# for healthmon to finish
+
+# Did we get errors?
+cat $tmp.healthmon >> $seqres.full
+filter_healer_errors < $tmp.healthmon
+
+_dmerror_cleanup
+
+status=0
+exit
tests/xfs/1878.out new file +10 -0
new file mode 100644
index 00000000000000..f64c440b1a6ed1
@@ -0,0 +1,10 @@
+QA output created by 1878
+Format and mount
+pwrite: Input/output error
+pread: Input/output error
+pread: Input/output error
+fsync: Input/output error
+VICTIM pos NUM len NUM: directio_write: Input/output error
+VICTIM pos NUM len NUM: directio_read: Input/output error
+VICTIM pos NUM len NUM: buffered_read: Input/output error
+VICTIM pos NUM len NUM: buffered_write: Input/output error
#100 lore raw

PATCH 4/13 xfs: set up common code for testing xfs_healer

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a bunch of common code so that we can test the xfs_healer daemon. Most of the changes here are to make it easier to manage the systemd service units for xfs_healer and xfs_scrub.

---
 common/config  |   14 +++++++
 common/rc      |    5 ++
 common/systemd |   32 ++++++++++++++++
 common/xfs     |  114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/802  |    4 +-
 5 files changed, 167 insertions(+), 2 deletions(-)
common/config +14 -0
index 1420e35ddfee42..8468a60081f50c 100644
@@ -161,6 +161,20 @@ export XFS_ADMIN_PROG="$(type -P xfs_admin)"
 export XFS_GROWFS_PROG=$(type -P xfs_growfs)
 export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)"
 export XFS_SCRUB_PROG="$(type -P xfs_scrub)"
+
+XFS_HEALER_PROG="$(type -P xfs_healer)"
+XFS_HEALER_START_PROG="$(type -P xfs_healer_start)"
+
+# If not found, try the ones installed in libexec
+if [ ! -x "$XFS_HEALER_PROG" ] && [ -e /usr/libexec/xfsprogs/xfs_healer ]; then
+	XFS_HEALER_PROG=/usr/libexec/xfsprogs/xfs_healer
+fi
+if [ ! -x "$XFS_HEALER_START_PROG" ] && [ -e /usr/libexec/xfsprogs/xfs_healer_start ]; then
+	XFS_HEALER_START_PROG=/usr/libexec/xfsprogs/xfs_healer_start
+fi
+export XFS_HEALER_PROG
+export XFS_HEALER_START_PROG
+
 export XFS_PARALLEL_REPAIR_PROG="$(type -P xfs_prepair)"
 export XFS_PARALLEL_REPAIR64_PROG="$(type -P xfs_prepair64)"
 export __XFSDUMP_PROG="$(type -P xfsdump)"
common/rc +5 -0
index 38d4b500b3b51f..91db0fb09da891 100644
@@ -3026,6 +3026,11 @@ _require_xfs_io_command()
 	"label")
 		testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1`
 		;;
+	"verifymedia")
+		testio=`$XFS_IO_PROG -x -c "verifymedia $* 0 0" 2>&1`
+		echo $testio | grep -q "invalid option" && \
+			_notrun "xfs_io $command support is missing"
+		;;
 	"open")
 		# -c "open $f" is broken in xfs_io <= 4.8. Along with the fix,
 		# a new -C flag was introduced to execute one shot commands.
common/systemd +32 -0
index b2e24f267b2d93..b4c77c78a8da44 100644
@@ -44,6 +44,18 @@ _systemd_unit_active() {
 	test "$(systemctl is-active "$1")" = "active"
 }
 
+# Wait for up to a certain number of seconds for a service to reach inactive
+# state.
+_systemd_unit_wait() {
+	local svcname="$1"
+	local timeout="${2:-30}"
+
+	for ((i = 0; i < (timeout * 2); i++)); do
+		test "$(systemctl is-active "$svcname")" = "inactive" && break
+		sleep 0.5
+	done
+}
+
 _require_systemd_unit_active() {
 	_require_systemd_unit_defined "$1"
 	_systemd_unit_active "$1" || \
@@ -71,3 +83,23 @@ _systemd_unit_status() {
 	_systemd_installed || return 1
 	systemctl status "$1"
 }
+
+# Start a running systemd unit
+_systemd_unit_start() {
+	systemctl start "$1"
+}
+# Stop a running systemd unit
+_systemd_unit_stop() {
+	systemctl stop "$1"
+}
+
+# Mask or unmask a running systemd unit
+_systemd_unit_mask() {
+	systemctl mask "$1"
+}
+_systemd_unit_unmask() {
+	systemctl unmask "$1"
+}
+_systemd_unit_masked() {
+	systemctl status "$1" 2>/dev/null | grep -q 'Loaded: masked'
+}
common/xfs +114 -0

117 lines not on this page · raw message

tests/xfs/802 +2 -2
index fc4767acb66a55..18312b15b645bd 100755
@@ -105,8 +105,8 @@ run_scrub_service() {
 }
 
 echo "Scrub Scratch FS"
-scratch_path=$(systemd-escape --path "$SCRATCH_MNT")
-run_scrub_service xfs_scrub@$scratch_path
+svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")"
+run_scrub_service "$svc"
 find_scrub_trace "$SCRATCH_MNT"
 
 # Remove the xfs_scrub_all media scan stamp directory (if specified) because we
#101 lore raw

PATCH 5/13 xfs: test xfs_healer's event handling

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that xfs_healer can handle every type of event that the kernel can throw at it by initiating a full scrub of a test filesystem.

---
 tests/xfs/1882     |   44 ++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1882.out |    2 ++
 2 files changed, 46 insertions(+)
 create mode 100755 tests/xfs/1882
 create mode 100644 tests/xfs/1882.out
tests/xfs/1882 new file +44 -0
new file mode 100755
index 00000000000000..2fb4589418401e
@@ -0,0 +1,44 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1882
+#
+# Make sure that xfs_healer correctly handles all the reports that it gets
+# from the kernel.  We simulate this by using the --everything mode so we get
+# all the events, not just the sickness reports.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing
+
+. ./common/filter
+. ./common/fuzzy
+. ./common/systemd
+. ./common/populate
+
+_require_scrub
+_require_xfs_io_command "scrub"		# online check support
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_scratch
+
+# Does this fs support health monitoring?
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+_require_xfs_healer $SCRATCH_MNT
+_scratch_unmount
+
+# Create a sample fs with all the goodies
+_scratch_populate_cached nofill &>> $seqres.full
+_scratch_mount
+
+_scratch_invoke_xfs_healer "$tmp.healer" --everything
+
+# Run scrub to make some noise
+_scratch_scrub -b -n >> $seqres.full
+
+_scratch_kill_xfs_healer
+cat $tmp.healer >> $seqres.full
+
+echo Silence is golden
+status=0
+exit
tests/xfs/1882.out new file +2 -0
new file mode 100644
index 00000000000000..9b31ccb735cabd
@@ -0,0 +1,2 @@
+QA output created by 1882
+Silence is golden
#102 lore raw

PATCH 6/13 xfs: test xfs_healer can fix a filesystem

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that xfs_healer can actually fix an injected metadata corruption.

---
 tests/xfs/1884     |   89 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1884.out |    2 +
 2 files changed, 91 insertions(+)
 create mode 100755 tests/xfs/1884
 create mode 100644 tests/xfs/1884.out
tests/xfs/1884 new file +89 -0
new file mode 100755
index 00000000000000..1fa6457ad25203
@@ -0,0 +1,89 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1884
+#
+# Ensure that autonomous self healing fixes the filesystem correctly.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing
+
+. ./common/filter
+. ./common/fuzzy
+. ./common/systemd
+
+_require_scrub
+_require_xfs_io_command "repair"	# online repair support
+_require_xfs_db_command "blocktrash"
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_command "$XFS_PROPERTY_PROG" "xfs_property"
+_require_scratch
+
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+
+_xfs_has_feature $SCRATCH_MNT rmapbt || \
+	_notrun "reverse mapping required to test directory auto-repair"
+_xfs_has_feature $SCRATCH_MNT parent || \
+	_notrun "parent pointers required to test directory auto-repair"
+_require_xfs_healer $SCRATCH_MNT --repair
+
+# Configure the filesystem for automatic repair of the filesystem.
+$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full
+
+# Create a largeish directory
+dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT")
+echo testdata > $SCRATCH_MNT/a
+mkdir -p "$SCRATCH_MNT/some/victimdir"
+for ((i = 0; i < (dblksz / 255); i++)); do
+	fname="$(printf "%0255d" "$i")"
+	ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname
+done
+
+# Did we get at least two dir blocks?
+dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir)
+test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory"
+
+# Break the directory, remount filesystem
+_scratch_unmount
+_scratch_xfs_db -x \
+	-c 'path /some/victimdir' \
+	-c 'bmap' \
+	-c 'dblock 1' \
+	-c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full
+_scratch_mount
+
+_scratch_invoke_xfs_healer "$tmp.healer" --repair
+
+# Access the broken directory to trigger a repair, then poll the directory
+# for 5 seconds to see if it gets fixed without us needing to intervene.
+ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+_filter_scratch < $tmp.err
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "try $try saw corruption" >> $seqres.full
+	sleep 0.1
+	ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "try $try no longer saw corruption or gave up" >> $seqres.full
+_filter_scratch < $tmp.err
+
+# List the dirents of /victimdir to see if it stops reporting corruption
+ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "retry $try still saw corruption" >> $seqres.full
+	sleep 0.1
+	ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "retry $try no longer saw corruption or gave up" >> $seqres.full
+
+# Unmount to kill the healer
+_scratch_kill_xfs_healer
+cat $tmp.healer >> $seqres.full
+
+status=0
+exit
tests/xfs/1884.out new file +2 -0
new file mode 100644
index 00000000000000..929e33da01f92c
@@ -0,0 +1,2 @@
+QA output created by 1884
+ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning
#103 lore raw

PATCH 7/13 xfs: test xfs_healer can report file I/O errors

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that xfs_healer can actually report file I/O errors.

---
 tests/xfs/1896     |  210 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1896.out |   21 +++++
 2 files changed, 231 insertions(+)
 create mode 100755 tests/xfs/1896
 create mode 100644 tests/xfs/1896.out
tests/xfs/1896 new file +210 -0
new file mode 100755
index 00000000000000..911e1d5ee8a576
@@ -0,0 +1,210 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1896
+#
+# Check that xfs_healer can report file IO errors.
+
+. ./common/preamble
+_begin_fstest auto quick scrub eio selfhealing
+
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	_dmerror_cleanup
+}
+
+# Import common functions.
+. ./common/fuzzy
+. ./common/filter
+. ./common/dmerror
+. ./common/systemd
+
+_require_scratch
+_require_scrub
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_dm_target error
+_require_no_xfs_always_cow	# no out of place writes
+
+# Ignore everything from the healer except for the four IO error log messages.
+# Strip out file handle and range information because the blocksize can vary.
+# Writeback and readahead can trigger multiple error messages due to retries,
+# hence the uniq.
+filter_healer_errors() {
+	_filter_scratch | \
+		grep -E '(buffered|directio)' | \
+		sed \
+		    -e 's/ino [0-9]*/ino NUM/g' \
+		    -e 's/gen 0x[0-9a-f]*/gen NUM/g' \
+		    -e 's/pos [0-9]*/pos NUM/g' \
+		    -e 's/len [0-9]*/len NUM/g' \
+		    -e 's|SCRATCH_MNT/a|VICTIM|g' \
+		    -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' | \
+		sort | \
+		uniq
+}
+
+_scratch_mkfs >> $seqres.full
+
+#
+# The dm-error map added by this test doesn't work on zoned devices because
+# table sizes need to be aligned to the zone size, and even for zoned on
+# conventional this test will get confused because of the internal RT device.
+#
+# That check requires a mounted file system, so do a dummy mount before setting
+# up DM.
+#
+_scratch_mount
+_require_xfs_scratch_non_zoned
+_require_xfs_healer $SCRATCH_MNT
+_scratch_unmount
+
+_dmerror_init
+_dmerror_mount >> $seqres.full 2>&1
+
+# Write a file with 4 file blocks worth of data, figure out the LBA to target
+victim=$SCRATCH_MNT/a
+file_blksz=$(_get_file_block_size $SCRATCH_MNT)
+$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full
+unset errordev
+
+awk_len_prog='{print $6}'
+if _xfs_is_realtime_file $victim; then
+	if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then
+		awk_len_prog='{print $4}'
+	fi
+	errordev="RT"
+fi
+bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")"
+echo "$errordev:$bmap_str" >> $seqres.full
+
+phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')"
+len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")"
+
+fs_blksz=$(_get_block_size $SCRATCH_MNT)
+echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full
+kernel_sectors_per_fs_block=$((fs_blksz / 512))
+
+# Did we get at least 4 fs blocks worth of extent?
+min_len_sectors=$(( 4 * kernel_sectors_per_fs_block ))
+test "$len" -lt $min_len_sectors && \
+	_fail "could not format a long enough extent on an empty fs??"
+
+phys_start=$(echo "$phys" | sed -e 's/\.\..*//g')
+
+echo "$errordev:$phys:$len:$fs_blksz:$phys_start" >> $seqres.full
+echo "victim file:" >> $seqres.full
+od -tx1 -Ad -c $victim >> $seqres.full
+
+# Set the dmerror table so that all IO will pass through.
+_dmerror_reset_table
+
+cat >> $seqres.full << ENDL
+dmerror before:
+$DMERROR_TABLE
+$DMERROR_RTTABLE
+<end table>
+ENDL
+
+# All sector numbers that we feed to the kernel must be in units of 512b, but
+# they also must be aligned to the device's logical block size.
+logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV`
+kernel_sectors_per_device_lba=$((logical_block_size / 512))
+
+# Mark as bad one of the device LBAs in the middle of the extent.  Target the
+# second LBA of the third block of the four-block file extent that we allocated
+# earlier, but without overflowing into the fourth file block.
+bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) ))
+bad_len=$kernel_sectors_per_device_lba
+if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then
+	bad_sector=$((bad_sector + kernel_sectors_per_device_lba))
+fi
+if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then
+	echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size"
+fi
+
+# Remount to flush the page cache, start the healer, and make the LBA bad
+_dmerror_unmount
+_dmerror_mount
+
+_scratch_invoke_xfs_healer "$tmp.healer"
+
+_dmerror_mark_range_bad $bad_sector $bad_len $errordev
+
+cat >> $seqres.full << ENDL
+dmerror after marking bad:
+$DMERROR_TABLE
+$DMERROR_RTTABLE
+<end table>
+ENDL
+
+_dmerror_load_error_table
+
+# See if buffered reads pick it up
+echo "Try buffered read"
+$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full
+
+# See if directio reads pick it up
+echo "Try directio read"
+$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full
+
+# See if directio writes pick it up
+echo "Try directio write"
+$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full
+
+# See if buffered writes pick it up
+echo "Try buffered write"
+$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full
+
+# Now mark the bad range good so that unmount won't fail due to IO errors.
+echo "Fix device"
+_dmerror_mark_range_good $bad_sector $bad_len $errordev
+_dmerror_load_error_table
+
+cat >> $seqres.full << ENDL
+dmerror after marking good:
+$DMERROR_TABLE
+$DMERROR_RTTABLE
+<end table>
+ENDL
+
+# Unmount filesystem to start fresh
+echo "Kill healer"
+_scratch_kill_xfs_healer _dmerror_unmount
+cat $tmp.healer >> $seqres.full
+cat $tmp.healer | filter_healer_errors
+
+# Start the healer again so that can verify that the errors don't persist after
+# we flip back to the good dm table.
+echo "Remount and restart healer"
+_dmerror_mount
+_scratch_invoke_xfs_healer "$tmp.healer"
+
+# See if buffered reads pick it up
+echo "Try buffered read again"
+$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full
+
+# See if directio reads pick it up
+echo "Try directio read again"
+$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full
+
+# See if directio writes pick it up
+echo "Try directio write again"
+$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full
+
+# See if buffered writes pick it up
+echo "Try buffered write again"
+$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full
+
+# Unmount fs to kill healer, then wait for it to finish
+echo "Kill healer again"
+_scratch_kill_xfs_healer _dmerror_unmount
+cat $tmp.healer >> $seqres.full
+cat $tmp.healer | filter_healer_errors
+
+# success, all done
+status=0
+exit
tests/xfs/1896.out new file +21 -0
new file mode 100644
index 00000000000000..1378d4fad44522
@@ -0,0 +1,21 @@
+QA output created by 1896
+Try buffered read
+pread: Input/output error
+Try directio read
+pread: Input/output error
+Try directio write
+pwrite: Input/output error
+Try buffered write
+fsync: Input/output error
+Fix device
+Kill healer
+VICTIM pos NUM len NUM: buffered_read: Input/output error
+VICTIM pos NUM len NUM: buffered_write: Input/output error
+VICTIM pos NUM len NUM: directio_read: Input/output error
+VICTIM pos NUM len NUM: directio_write: Input/output error
+Remount and restart healer
+Try buffered read again
+Try directio read again
+Try directio write again
+Try buffered write again
+Kill healer again
#104 lore raw

PATCH 8/13 xfs: test xfs_healer can report file media errors

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that xfs_healer can actually report media errors as found by the kernel.

---
 tests/xfs/1897     |  172 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1897.out |    7 ++
 2 files changed, 179 insertions(+)
 create mode 100755 tests/xfs/1897
 create mode 100755 tests/xfs/1897.out
tests/xfs/1897 new file +172 -0
new file mode 100755
index 00000000000000..4670c333a2d82c
@@ -0,0 +1,172 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1897
+#
+# Check that xfs_healer can report media errors.
+
+. ./common/preamble
+_begin_fstest auto quick scrub eio selfhealing
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	_dmerror_cleanup
+}
+
+. ./common/fuzzy
+. ./common/filter
+. ./common/dmerror
+. ./common/systemd
+
+_require_scratch
+_require_scrub
+_require_dm_target error
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_xfs_io_command verifymedia
+
+filter_healer() {
+	_filter_scratch | \
+		grep -E '(media failed|media error)' | \
+		sed \
+		    -e 's/datadev/DEVICE/g' \
+		    -e 's/rtdev/DEVICE/g' \
+		    -e 's/ino [0-9]*/ino NUM/g' \
+		    -e 's/gen 0x[0-9a-f]*/gen NUM/g' \
+		    -e 's/pos [0-9]*/pos NUM/g' \
+		    -e 's/len [0-9]*/len NUM/g' \
+		    -e 's/0x[0-9a-f]*/NUM/g' \
+		    -e 's|SCRATCH_MNT/a|VICTIM|g' \
+		    -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g'
+}
+
+filter_verify() {
+	sed -e 's/\([a-z]*dev\): verify error at offset \([0-9]*\) length \([0-9]*\)/DEVICE: verify error at offset XXX length XXX/g'
+}
+
+_scratch_mkfs >> $seqres.full
+
+# The dm-error map added by this test doesn't work on zoned devices because
+# table sizes need to be aligned to the zone size, and even for zoned on
+# conventional this test will get confused because of the internal RT device.
+#
+# That check requires a mounted file system, so do a dummy mount before setting
+# up DM.
+_scratch_mount
+_require_xfs_scratch_non_zoned
+_require_xfs_healer $SCRATCH_MNT
+_scratch_unmount
+
+_dmerror_init
+_dmerror_mount
+
+# Write a file with 4 file blocks worth of data, figure out the LBA to target
+victim=$SCRATCH_MNT/a
+file_blksz=$(_get_file_block_size $SCRATCH_MNT)
+$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full
+unset errordev
+verifymediadev="-d"
+
+awk_len_prog='{print $6}'
+if _xfs_is_realtime_file $victim; then
+	if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then
+		awk_len_prog='{print $4}'
+	fi
+	errordev="RT"
+	verifymediadev="-r"
+fi
+bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")"
+echo "$errordev:$bmap_str" >> $seqres.full
+
+phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')"
+len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")"
+
+fs_blksz=$(_get_block_size $SCRATCH_MNT)
+echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full
+kernel_sectors_per_fs_block=$((fs_blksz / 512))
+
+# Did we get at least 4 fs blocks worth of extent?
+min_len_sectors=$(( 4 * kernel_sectors_per_fs_block ))
+test "$len" -lt $min_len_sectors && \
+	_fail "could not format a long enough extent on an empty fs??"
+
+phys_start=$(echo "$phys" | sed -e 's/\.\..*//g')
+
+echo "$errordev:$phys:$len:$fs_blksz:$phys_start" >> $seqres.full
+echo "victim file:" >> $seqres.full
+od -tx1 -Ad -c $victim >> $seqres.full
+
+# Set the dmerror table so that all IO will pass through.
+_dmerror_reset_table
+
+cat >> $seqres.full << ENDL
+dmerror before:
+$DMERROR_TABLE
+$DMERROR_RTTABLE
+<end table>
+ENDL
+
+# All sector numbers that we feed to the kernel must be in units of 512b, but
+# they also must be aligned to the device's logical block size.
+logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV`
+kernel_sectors_per_device_lba=$((logical_block_size / 512))
+
+# Pretend as bad one of the device LBAs in the middle of the extent.  Target
+# the second LBA of the third block of the four-block file extent that we
+# allocated earlier, but without overflowing into the fourth file block.
+bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) ))
+bad_len=$kernel_sectors_per_device_lba
+if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then
+	bad_sector=$((bad_sector + kernel_sectors_per_device_lba))
+fi
+if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then
+	echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size"
+fi
+_dmerror_mark_range_bad $bad_sector $bad_len $errordev
+
+cat >> $seqres.full << ENDL
+dmerror after marking bad:
+$DMERROR_TABLE
+$DMERROR_RTTABLE
+<end table>
+ENDL
+
+_dmerror_load_error_table
+
+echo "Simulate media error"
+_scratch_invoke_xfs_healer "$tmp.healer"
+echo "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" >> $seqres.full
+$XFS_IO_PROG -x -c "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" $SCRATCH_MNT 2>&1 | filter_verify
+
+# Now mark the bad range good so that a retest shows no media failure.
+_dmerror_mark_range_good $bad_sector $bad_len $errordev
+_dmerror_load_error_table
+
+cat >> $seqres.full << ENDL
+dmerror after marking good:
+$DMERROR_TABLE
+$DMERROR_RTTABLE
+<end table>
+ENDL
+
+echo "No more media error"
+echo "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" >> $seqres.full
+$XFS_IO_PROG -x -c "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" $SCRATCH_MNT >> $seqres.full
+
+# Unmount filesystem to start fresh
+echo "Kill healer"
+_scratch_kill_xfs_healer _dmerror_unmount
+
+# filesystems without rmap do not translate media errors to lost file ranges
+# so fake the output
+_xfs_has_feature "$SCRATCH_DEV" rmapbt || \
+	echo "VICTIM pos 0 len 0: media failed" >> $tmp.healer
+
+cat $tmp.healer >> $seqres.full
+cat $tmp.healer | filter_healer
+
+# success, all done
+status=0
+exit
tests/xfs/1897.out new file +7 -0
new file mode 100755
index 00000000000000..1bb615c3119dce
@@ -0,0 +1,7 @@
+QA output created by 1897
+Simulate media error
+DEVICE: verify error at offset XXX length XXX: Input/output error
+No more media error
+Kill healer
+SCRATCH_MNT DEVICE daddr NUM bbcount NUM: media error
+VICTIM pos NUM len NUM: media failed
#105 lore raw

PATCH 9/13 xfs: test xfs_healer can report filesystem shutdowns

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that xfs_healer can actually report abnormal filesystem shutdowns.

---
 tests/xfs/1898     |   37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1898.out |    4 ++++
 2 files changed, 41 insertions(+)
 create mode 100755 tests/xfs/1898
 create mode 100755 tests/xfs/1898.out
tests/xfs/1898 new file +37 -0
new file mode 100755
index 00000000000000..2b6c72093e7021
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1898
+#
+# Check that xfs_healer can report filesystem shutdowns.
+
+. ./common/preamble
+_begin_fstest auto quick scrub eio selfhealing
+
+. ./common/fuzzy
+. ./common/filter
+. ./common/systemd
+
+_require_scratch_nocheck
+_require_scrub
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+_require_xfs_healer $SCRATCH_MNT
+$XFS_IO_PROG -f -c "pwrite -S 0x58 0 500k" -c "fsync" $victim >> $seqres.full
+
+echo "Start healer and shut down"
+_scratch_invoke_xfs_healer "$tmp.healer"
+_scratch_shutdown -f
+
+# Unmount filesystem to start fresh
+echo "Kill healer"
+_scratch_kill_xfs_healer
+cat $tmp.healer >> $seqres.full
+cat $tmp.healer | _filter_scratch | grep 'shut down'
+
+# success, all done
+status=0
+exit
tests/xfs/1898.out new file +4 -0
new file mode 100755
index 00000000000000..f71f848da810ce
@@ -0,0 +1,4 @@
+QA output created by 1898
+Start healer and shut down
+Kill healer
+SCRATCH_MNT: filesystem shut down due to forced unmount
#106 lore raw

PATCH 10/13 xfs: test xfs_healer can initiate full filesystem repairs

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that when xfs_healer can't perform a spot repair, it will actually start up xfs_scrub to perform a full scan and repair.

---
 tests/xfs/1899     |  108 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1899.out |    3 +
 2 files changed, 111 insertions(+)
 create mode 100755 tests/xfs/1899
 create mode 100644 tests/xfs/1899.out
tests/xfs/1899 new file +108 -0
new file mode 100755
index 00000000000000..5d35ca8265645f
@@ -0,0 +1,108 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1899
+#
+# Ensure that autonomous self healing works fixes the filesystem correctly
+# even if the spot repair doesn't work and it falls back to a full fsck.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing
+
+. ./common/filter
+. ./common/fuzzy
+. ./common/systemd
+
+_require_scrub
+_require_xfs_io_command "repair"	# online repair support
+_require_xfs_db_command "blocktrash"
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_command "$XFS_PROPERTY_PROG" "xfs_property"
+_require_scratch
+_require_systemd_unit_defined "[email protected]"
+
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+
+_xfs_has_feature $SCRATCH_MNT rmapbt || \
+	_notrun "reverse mapping required to test directory auto-repair"
+_xfs_has_feature $SCRATCH_MNT parent || \
+	_notrun "parent pointers required to test directory auto-repair"
+_require_xfs_healer $SCRATCH_MNT --repair
+
+filter_healer() {
+	_filter_scratch | \
+		grep 'Full repairs in progress' | \
+		uniq
+}
+
+# Configure the filesystem for automatic repair of the filesystem.
+$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full
+
+# Create a largeish directory
+dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT")
+echo testdata > $SCRATCH_MNT/a
+mkdir -p "$SCRATCH_MNT/some/victimdir"
+for ((i = 0; i < (dblksz / 255); i++)); do
+	fname="$(printf "%0255d" "$i")"
+	ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname
+done
+
+# Did we get at least two dir blocks?
+dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir)
+test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory"
+
+# Break the directory, remount filesystem
+_scratch_unmount
+_scratch_xfs_db -x \
+	-c 'path /some/victimdir' \
+	-c 'bmap' \
+	-c 'dblock 1' \
+	-c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' \
+	-c 'path /a' \
+	-c 'bmap -a' \
+	-c 'ablock 1' \
+	-c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' \
+	>> $seqres.full
+_scratch_mount
+
+_scratch_invoke_xfs_healer "$tmp.healer" --repair
+
+# Access the broken directory to trigger a repair, then poll the directory
+# for 5 seconds to see if it gets fixed without us needing to intervene.
+ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+_filter_scratch < $tmp.err
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "try $try saw corruption" >> $seqres.full
+	sleep 0.1
+	ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "try $try no longer saw corruption or gave up" >> $seqres.full
+_filter_scratch < $tmp.err
+
+# Wait for the background fixer to finish
+svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")"
+_systemd_unit_wait "$svc"
+
+# List the dirents of /victimdir and parent pointers of /a to see if they both
+# stop reporting corruption
+(ls $SCRATCH_MNT/some/victimdir ; $XFS_IO_PROG -c 'parent') > /dev/null 2> $tmp.err
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "retry $try still saw corruption" >> $seqres.full
+	sleep 0.1
+	(ls $SCRATCH_MNT/some/victimdir ; $XFS_IO_PROG -c 'parent') > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "retry $try no longer saw corruption or gave up" >> $seqres.full
+
+# Unmount to kill the healer
+_scratch_kill_xfs_healer
+cat $tmp.healer >> $seqres.full
+cat $tmp.healer | filter_healer
+
+status=0
+exit
tests/xfs/1899.out new file +3 -0
new file mode 100644
index 00000000000000..5345fd400f3627
@@ -0,0 +1,3 @@
+QA output created by 1899
+ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning
+SCRATCH_MNT: Full repairs in progress.
#107 lore raw

PATCH 11/13 xfs: test xfs_healer can follow mount moves

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that when xfs_healer needs to reopen a filesystem to repair it, it can still find the filesystem even if it has been mount --move'd.

---
 tests/xfs/1900     |  115 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1900.out |    2 +
 2 files changed, 117 insertions(+)
 create mode 100755 tests/xfs/1900
 create mode 100755 tests/xfs/1900.out
tests/xfs/1900 new file +115 -0
new file mode 100755
index 00000000000000..9a8f9fabd124ad
@@ -0,0 +1,115 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1900
+#
+# Ensure that autonomous self healing fixes the filesystem correctly even if
+# the original mount has moved somewhere else.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing
+
+. ./common/filter
+. ./common/fuzzy
+. ./common/systemd
+
+_cleanup()
+{
+	command -v _kill_fsstress &>/dev/null && _kill_fsstress
+	cd /
+	rm -r -f $tmp.*
+	if [ -n "$new_dir" ]; then
+		_unmount "$new_dir" &>/dev/null
+		rm -rf "$new_dir"
+	fi
+}
+
+_require_test
+_require_scrub
+_require_xfs_io_command "repair"	# online repair support
+_require_xfs_db_command "blocktrash"
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_command "$XFS_PROPERTY_PROG" "xfs_property"
+_require_scratch
+
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+
+_xfs_has_feature $SCRATCH_MNT rmapbt || \
+	_notrun "reverse mapping required to test directory auto-repair"
+_xfs_has_feature $SCRATCH_MNT parent || \
+	_notrun "parent pointers required to test directory auto-repair"
+_require_xfs_healer $SCRATCH_MNT --repair
+
+# Configure the filesystem for automatic repair of the filesystem.
+$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full
+
+# Create a largeish directory
+dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT")
+echo testdata > $SCRATCH_MNT/a
+mkdir -p "$SCRATCH_MNT/some/victimdir"
+for ((i = 0; i < (dblksz / 255); i++)); do
+	fname="$(printf "%0255d" "$i")"
+	ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname
+done
+
+# Did we get at least two dir blocks?
+dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir)
+test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory"
+
+# Break the directory, remount filesystem
+_scratch_unmount
+_scratch_xfs_db -x \
+	-c 'path /some/victimdir' \
+	-c 'bmap' \
+	-c 'dblock 1' \
+	-c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full
+_scratch_mount
+
+_scratch_invoke_xfs_healer "$tmp.healer" --repair
+
+# Move the scratch filesystem to a completely different mountpoint so that
+# we can test if the healer can find it again.
+new_dir=$TEST_DIR/moocow
+mkdir -p $new_dir
+_mount --bind $SCRATCH_MNT $new_dir
+_unmount $SCRATCH_MNT
+
+df -t xfs >> $seqres.full
+
+# Access the broken directory to trigger a repair, then poll the directory
+# for 5 seconds to see if it gets fixed without us needing to intervene.
+ls $new_dir/some/victimdir > /dev/null 2> $tmp.err
+_filter_scratch < $tmp.err | _filter_test_dir
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "try $try saw corruption" >> $seqres.full
+	sleep 0.1
+	ls $new_dir/some/victimdir > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "try $try no longer saw corruption or gave up" >> $seqres.full
+_filter_scratch < $tmp.err | _filter_test_dir
+
+# List the dirents of /victimdir to see if it stops reporting corruption
+ls $new_dir/some/victimdir > /dev/null 2> $tmp.err
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "retry $try still saw corruption" >> $seqres.full
+	sleep 0.1
+	ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "retry $try no longer saw corruption or gave up" >> $seqres.full
+
+new_dir_unmount() {
+	_unmount $new_dir
+}
+
+# Unmount to kill the healer
+_scratch_kill_xfs_healer new_dir_unmount
+cat $tmp.healer >> $seqres.full
+
+status=0
+exit
tests/xfs/1900.out new file +2 -0
new file mode 100755
index 00000000000000..604c9eb5eb10f4
@@ -0,0 +1,2 @@
+QA output created by 1900
+ls: reading directory 'TEST_DIR/moocow/some/victimdir': Structure needs cleaning
#108 lore raw

PATCH 12/13 xfs: test xfs_healer wont repair the wrong filesystem

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that when xfs_healer needs to reopen a filesystem to repair it, it won't latch on to another xfs filesystem that has been mounted atop the same mountpoint.

---
 tests/xfs/1901     |  137 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1901.out |    2 +
 2 files changed, 139 insertions(+)
 create mode 100755 tests/xfs/1901
 create mode 100755 tests/xfs/1901.out
tests/xfs/1901 new file +137 -0
new file mode 100755
index 00000000000000..c92dcf9a3b3d48
@@ -0,0 +1,137 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1901
+#
+# Ensure that autonomous self healing won't fix the wrong filesystem if a
+# snapshot of the original filesystem is now mounted on the same directory as
+# the original.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing
+
+. ./common/filter
+. ./common/fuzzy
+. ./common/systemd
+
+_cleanup()
+{
+	command -v _kill_fsstress &>/dev/null && _kill_fsstress
+	cd /
+	rm -r -f $tmp.*
+	test -e "$mntpt" && _unmount "$mntpt" &>/dev/null
+	test -e "$mntpt" && _unmount "$mntpt" &>/dev/null
+	test -e "$loop1" && _destroy_loop_device "$loop1"
+	test -e "$loop2" && _destroy_loop_device "$loop2"
+	test -e "$testdir" && rm -r -f "$testdir"
+}
+
+_require_test
+_require_scrub
+_require_xfs_io_command "repair"	# online repair support
+_require_xfs_db_command "blocktrash"
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_command "$XFS_PROPERTY_PROG" "xfs_property"
+
+testdir=$TEST_DIR/$seq
+mntpt=$testdir/mount
+disk1=$testdir/disk1
+disk2=$testdir/disk2
+
+mkdir -p "$mntpt"
+$XFS_IO_PROG -f -c "truncate 300m" $disk1
+$XFS_IO_PROG -f -c "truncate 300m" $disk2
+loop1="$(_create_loop_device "$disk1")"
+
+filter_mntpt() {
+	sed -e "s|$mntpt|MNTPT|g"
+}
+
+_mkfs_dev "$loop1" >> $seqres.full
+_mount "$loop1" "$mntpt" || _notrun "cannot mount victim filesystem"
+
+_xfs_has_feature $mntpt rmapbt || \
+	_notrun "reverse mapping required to test directory auto-repair"
+_xfs_has_feature $mntpt parent || \
+	_notrun "parent pointers required to test directory auto-repair"
+_require_xfs_healer $mntpt --repair
+
+# Configure the filesystem for automatic repair of the filesystem.
+$XFS_PROPERTY_PROG $mntpt set autofsck=repair >> $seqres.full
+
+# Create a largeish directory
+dblksz=$(_xfs_get_dir_blocksize "$mntpt")
+echo testdata > $mntpt/a
+mkdir -p "$mntpt/some/victimdir"
+for ((i = 0; i < (dblksz / 255); i++)); do
+	fname="$(printf "%0255d" "$i")"
+	ln $mntpt/a $mntpt/some/victimdir/$fname
+done
+
+# Did we get at least two dir blocks?
+dirsize=$(stat -c '%s' $mntpt/some/victimdir)
+test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory"
+
+# Clone the fs, break the directory, remount filesystem
+_unmount "$mntpt"
+
+cp --sparse=always "$disk1" "$disk2" || _fail "cannot copy disk1"
+loop2="$(_create_loop_device_like_bdev "$disk2" "$loop1")"
+
+$XFS_DB_PROG "$loop1" -x \
+	-c 'path /some/victimdir' \
+	-c 'bmap' \
+	-c 'dblock 1' \
+	-c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full
+_mount "$loop1" "$mntpt" || _fail "cannot mount broken fs"
+
+_invoke_xfs_healer "$mntpt" "$tmp.healer" --repair
+
+# Stop the healer process so that it can't read error events while we do some
+# shenanigans.
+test -n "$XFS_HEALER_PID" || _fail "nobody set XFS_HEALER_PID?"
+kill -STOP $XFS_HEALER_PID
+
+
+echo "LOG $XFS_HEALER_PID SO FAR:" >> $seqres.full
+cat $tmp.healer >> $seqres.full
+
+# Access the broken directory to trigger a repair event, which will not yet be
+# processed.
+ls $mntpt/some/victimdir > /dev/null 2> $tmp.err
+filter_mntpt < $tmp.err
+
+ps auxfww | grep xfs_healer >> $seqres.full
+
+echo "LOG AFTER TRYING TO POKE:" >> $seqres.full
+cat $tmp.healer >> $seqres.full
+
+# Mount the clone filesystem to the same mountpoint so that the healer cannot
+# actually reopen it to perform repairs.
+_mount "$loop2" "$mntpt" -o nouuid || _fail "cannot mount decoy fs"
+
+grep -w xfs /proc/mounts >> $seqres.full
+
+# Continue the healer process so it can handle events now.  Wait a few seconds
+# while it fails to reopen disk1's mount point to repair things.
+kill -CONT $XFS_HEALER_PID
+sleep 2
+
+new_dir_unmount() {
+	_unmount "$mntpt"
+	_unmount "$mntpt"
+}
+
+# Unmount to kill the healer
+_kill_xfs_healer new_dir_unmount
+echo "LOG AFTER FAILURE" >> $seqres.full
+cat $tmp.healer >> $seqres.full
+
+# Did the healer log complaints about not being able to reopen the mountpoint
+# to enact repairs?
+grep -q 'Stale file handle' $tmp.healer || \
+	echo "Should have seen stale file handle complaints"
+
+status=0
+exit
tests/xfs/1901.out new file +2 -0
new file mode 100755
index 00000000000000..ff83e03725307a
@@ -0,0 +1,2 @@
+QA output created by 1901
+ls: reading directory 'MNTPT/some/victimdir': Structure needs cleaning
#109 lore raw

PATCH 13/13 xfs: test xfs_healer background service

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that when xfs_healer can monitor and repair filesystems when it's running as a systemd service, which is the intended usage model.

---
 tests/xfs/1902     |  152 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1902.out |    2 +
 2 files changed, 154 insertions(+)
 create mode 100755 tests/xfs/1902
 create mode 100755 tests/xfs/1902.out
tests/xfs/1902 new file +152 -0
new file mode 100755
index 00000000000000..d327995df8c5b0
@@ -0,0 +1,152 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1902
+#
+# Ensure that autonomous self healing fixes the filesystem correctly when
+# running in a systemd service
+#
+# unreliable_in_parallel: this test runs the xfs_healer systemd service, which
+# cannot be isolated to a specific testcase with the way check-parallel is
+# implemented.
+#
+. ./common/preamble
+_begin_fstest auto selfhealing unreliable_in_parallel
+
+_cleanup()
+{
+	cd /
+	if [ -n "$new_svcfile" ]; then
+		rm -f "$new_svcfile"
+		systemctl daemon-reload
+	fi
+	rm -r -f $tmp.*
+}
+
+. ./common/filter
+. ./common/fuzzy
+. ./common/systemd
+
+_require_systemd_is_running
+_require_systemd_unit_defined [email protected]
+_require_scrub
+_require_xfs_io_command "repair"	# online repair support
+_require_xfs_db_command "blocktrash"
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_command "$XFS_PROPERTY_PROG" "xfs_property"
+_require_scratch
+
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+
+_xfs_has_feature $SCRATCH_MNT rmapbt || \
+	_notrun "reverse mapping required to test directory auto-repair"
+_xfs_has_feature $SCRATCH_MNT parent || \
+	_notrun "parent pointers required to test directory auto-repair"
+_require_xfs_healer $SCRATCH_MNT --repair
+
+# Configure the filesystem for automatic repair of the filesystem.
+$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full
+
+# Create a largeish directory
+dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT")
+echo testdata > $SCRATCH_MNT/a
+mkdir -p "$SCRATCH_MNT/some/victimdir"
+for ((i = 0; i < (dblksz / 255); i++)); do
+	fname="$(printf "%0255d" "$i")"
+	ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname
+done
+
+# Did we get at least two dir blocks?
+dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir)
+test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory"
+
+# Break the directory
+_scratch_unmount
+_scratch_xfs_db -x \
+	-c 'path /some/victimdir' \
+	-c 'bmap' \
+	-c 'dblock 1' \
+	-c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full
+
+# Find the existing xfs_healer@ service definition, figure out where we're
+# going to land our test-specific override
+orig_svcfile="$(_systemd_unit_path "[email protected]")"
+test -f "$orig_svcfile" || \
+	_notrun "cannot find xfs_healer@ service file"
+
+new_svcdir="$(_systemd_runtime_dir)"
+test -d "$new_svcdir" || \
+	_notrun "cannot find runtime systemd service dir"
+
+# We need to make some local mods to the xfs_healer@ service definition
+# so we fork it and create a new service just for this test.
+new_healer_template="[email protected]"
+new_healer_svc="$(_xfs_systemd_svcname "$new_healer_template" "$SCRATCH_MNT")"
+_systemd_unit_status "$new_healer_svc" 2>&1 | \
+	grep -E -q '(could not be found|Loaded: not-found)' || \
+	_notrun "systemd service \"$new_healer_svc\" found, will not mess with this"
+
+new_svcfile="$new_svcdir/$new_healer_template"
+cp "$orig_svcfile" "$new_svcfile"
+
+# Pick up all the CLI args except for --repair and --no-autofsck because we're
+# going to force it to --autofsck below
+execargs="$(grep '^ExecStart=' $new_svcfile | \
+	    sed -e 's/^ExecStart=\S*//g' \
+	        -e 's/--no-autofsck//g' \
+		-e 's/--repair//g')"
+sed -e '/ExecStart=/d' -e '/BindPaths=/d' -e '/ExecCondition=/d' -i $new_svcfile
+cat >> "$new_svcfile" << ENDL
+
+[Service]
+ExecCondition=$XFS_HEALER_PROG --supported %f
+ExecStart=$XFS_HEALER_PROG $execargs
+ENDL
+_systemd_reload
+
+# Emit the results of our editing to the full log.
+systemctl cat "$new_healer_svc" >> $seqres.full
+
+# Remount, with service activation
+_scratch_mount
+
+old_healer_svc="$(_xfs_healer_svcname "$SCRATCH_MNT")"
+_systemd_unit_stop "$old_healer_svc" &>> $seqres.full
+_systemd_unit_start "$new_healer_svc" &>> $seqres.full
+
+_systemd_unit_status "$new_healer_svc" 2>&1 | grep -q 'Active: active' || \
+	echo "systemd service \"$new_healer_svc\" not running??"
+
+# Access the broken directory to trigger a repair, then poll the directory
+# for 5 seconds to see if it gets fixed without us needing to intervene.
+ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+_filter_scratch < $tmp.err
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "try $try saw corruption" >> $seqres.full
+	sleep 0.1
+	ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "try $try no longer saw corruption or gave up" >> $seqres.full
+_filter_scratch < $tmp.err
+
+# List the dirents of /victimdir to see if it stops reporting corruption
+ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+try=0
+while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do
+	echo "retry $try still saw corruption" >> $seqres.full
+	sleep 0.1
+	ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err
+	try=$((try + 1))
+done
+echo "retry $try no longer saw corruption or gave up" >> $seqres.full
+
+# Unmount to kill the healer
+_scratch_kill_xfs_healer
+journalctl -u "$new_healer_svc" >> $seqres.full
+
+status=0
+exit
tests/xfs/1902.out new file +2 -0
new file mode 100755
index 00000000000000..84f9b9e50e1e02
@@ -0,0 +1,2 @@
+QA output created by 1902
+ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning
#110 lore raw

[PATCH 14/13] xfs: test xfs_healer startup service

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that xfs_healer_start can actually start up xfs_healer service instances when a filesystem is mounted.

---
 tests/xfs/1903     |  124 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1903.out |    6 +++
 2 files changed, 130 insertions(+)
 create mode 100755 tests/xfs/1903
 create mode 100644 tests/xfs/1903.out
tests/xfs/1903 new file +124 -0
new file mode 100755
index 00000000000000..d71d75a6af3f9d
@@ -0,0 +1,124 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1903
+#
+# Check that the xfs_healer startup service starts the per-mount xfs_healer
+# service for the scratch filesystem.  IOWs, this is basic testing for the
+# xfs_healer systemd background services.
+#
+
+# unreliable_in_parallel: this appears to try to run healer services on all
+# mounted filesystems - that's a problem when there are a hundred other test
+# filesystems mounted running other tests...
+
+. ./common/preamble
+_begin_fstest auto selfhealing unreliable_in_parallel
+
+_cleanup()
+{
+	cd /
+	test -n "$new_healerstart_svc" &&
+		_systemd_unit_stop "$new_healerstart_svc"
+	test -n "$was_masked" && \
+		_systemd_unit_mask "$healer_svc" &>> $seqres.full
+	if [ -n "$new_svcfile" ]; then
+		rm -f "$new_svcfile"
+		systemctl daemon-reload
+	fi
+	rm -r -f $tmp.*
+}
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/systemd
+
+_require_systemd_is_running
+_require_systemd_unit_defined [email protected]
+_require_systemd_unit_defined xfs_healer_start.service
+_require_scratch
+_require_scrub
+_require_xfs_io_command "scrub"
+_require_xfs_spaceman_command "health"
+_require_populate_commands
+_require_command "$XFS_HEALER_PROG" "xfs_healer"
+_require_command $ATTR_PROG "attr"
+
+_xfs_skip_online_rebuild
+_xfs_skip_offline_rebuild
+
+orig_svcfile="$(_systemd_unit_path "xfs_healer_start.service")"
+test -f "$orig_svcfile" || \
+	_notrun "cannot find xfs_healer_start service file"
+
+new_svcdir="$(_systemd_runtime_dir)"
+test -d "$new_svcdir" || \
+	_notrun "cannot find runtime systemd service dir"
+
+# We need to make some local mods to the xfs_healer_start service definition
+# so we fork it and create a new service just for this test.
+new_healerstart_svc="xfs_healer_start_fstest.service"
+_systemd_unit_status "$new_healerstart_svc" 2>&1 | \
+	grep -E -q '(could not be found|Loaded: not-found)' || \
+	_notrun "systemd service \"$new_healerstart_svc\" found, will not mess with this"
+
+find_healer_trace() {
+	local path="$1"
+
+	sleep 2		# wait for delays in startup
+	$XFS_HEALER_PROG --supported "$path" 2>&1 | grep -q 'already running' || \
+		echo "cannot find evidence that xfs_healer is running for $path"
+}
+
+echo "Format and populate"
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+_require_xfs_healer $SCRATCH_MNT
+
+# Configure the filesystem for background checks of the filesystem.
+$ATTR_PROG -R -s xfs:autofsck -V check $SCRATCH_MNT >> $seqres.full
+
+was_masked=
+healer_svc="$(_xfs_healer_svcname "$SCRATCH_MNT")"
+
+# Preserve the xfs_healer@ mask state -- we don't want this permanently
+# changing global state.
+if _systemd_unit_masked "$healer_svc"; then
+	_systemd_unit_unmask "$healer_svc" &>> $seqres.full
+	was_masked=1
+fi
+
+echo "Start healer on scratch FS"
+_systemd_unit_start "$healer_svc"
+find_healer_trace "$SCRATCH_MNT"
+_systemd_unit_stop "$healer_svc"
+
+new_svcfile="$new_svcdir/$new_healerstart_svc"
+cp "$orig_svcfile" "$new_svcfile"
+
+sed -e '/ExecStart=/d' -e '/BindPaths=/d' -e '/ExecCondition=/d' -i $new_svcfile
+cat >> "$new_svcfile" << ENDL
+[Service]
+ExecCondition=$XFS_HEALER_START_PROG --supported
+ExecStart=$XFS_HEALER_START_PROG
+ENDL
+_systemd_reload
+
+# Emit the results of our editing to the full log.
+systemctl cat "$new_healerstart_svc" >> $seqres.full
+
+echo "Start healer for everything"
+_systemd_unit_start "$new_healerstart_svc"
+find_healer_trace "$SCRATCH_MNT"
+
+echo "Restart healer for scratch FS"
+_scratch_cycle_mount
+find_healer_trace "$SCRATCH_MNT"
+
+echo "Healer testing done" | tee -a $seqres.full
+
+# success, all done
+status=0
+exit
tests/xfs/1903.out new file +6 -0
new file mode 100644
index 00000000000000..07810f60ca10c6
@@ -0,0 +1,6 @@
+QA output created by 1903
+Format and populate
+Start healer on scratch FS
+Start healer for everything
+Restart healer for scratch FS
+Healer testing done