xfsprogs: autonomous self healing of filesystems
PATCH v9 1/1 xfsprogs: autonomous self healing of filesystems
Hi all,
This patchset contains the userspace and QA changes (xfs_healer) needed to put to use all the new kernel functionality to deliver live information about filesystem health events (xfs_healthmon.c) to userspace.
In userspace, we create a new daemon program that will read the event objects and initiate repairs automatically. This daemon is managed entirely by systemd and will not block unmounting of the filesystem unless repairs are ongoing. They are auto-started by a starter service that uses fanotify.
When the patchsets under this cover letter are merged, online fsck for XFS will at long last be fully feature complete. The passive scan parts have been done since mid-2024, this final part adds proactive repair.
These patches still need some review, please help if you can. I will get to the RWF_WRITETHOUGH series tomorrow.
[PATCHSET v9] xfsprogs: autonomous self healing of filesystems
[PATCH 03/28] libfrog: add support code for starting systemd services [PATCH 05/28] libfrog: add wrappers for listmount and statmount [PATCH 13/28] xfs_healer: create a per-mount background monitoring [PATCH 14/28] xfs_healer: create a service to start the per-mount [PATCH 19/28] xfs_healer: use statmount to find moved filesystems [PATCH 20/28] xfs_healer: validate that repair fds point to the [PATCH 22/28] xfs_scrub: use the verify media ioctl during phase 6 if [PATCH 24/28] xfs_scrub: print systemd service names [PATCH 25/28] xfs_io: add listmount and statmount commands [PATCH 26/28] mkfs: enable online repair if all backrefs are enabled [PATCH 27/28] debian/control: listify the build dependencies [PATCH 28/28] debian: enable xfs_healer on the root filesystem by
[PATCHSET v9 1/2] fstests: test generic file IO error reporting
[PATCH 1/1] generic: test fsnotify filesystem error reporting
[PATCHSET v9 2/2] fstests: autonomous self healing of filesystems
[PATCH 01/14] xfs: test health monitoring code
[PATCH 02/14] xfs: test for metadata corruption error reporting via [PATCH 03/14] xfs: test io error reporting via healthmon [PATCH 04/14] xfs: set up common code for testing xfs_healer [PATCH 05/14] xfs: test xfs_healer's event handling
[PATCH 06/14] xfs: test xfs_healer can fix a filesystem [PATCH 07/14] xfs: test xfs_healer can report file I/O errors [PATCH 08/14] xfs: test xfs_healer can report file media errors [PATCH 09/14] xfs: test xfs_healer can report filesystem shutdowns [PATCH 10/14] xfs: test xfs_healer can initiate full filesystem [PATCH 11/14] xfs: test xfs_healer can follow mount moves [PATCH 12/14] xfs: test xfs_healer wont repair the wrong filesystem [PATCH 13/14] xfs: test xfs_healer background service [PATCH 14/14] xfs: test xfs_healer startup service
v9: reorg listmount/statmount, use it to find moved mounts, improve the
commit messages and documentation
v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
and filter bio status flags better
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status
--D
PATCH v9 1/1 xfsprogs: autonomous self healing of filesystems
Hi all,
This patchset builds new functionality to deliver live information about filesystem health events to userspace. This is done by creating an anonymous file that can be read() for events by userspace programs. Events are captured by hooking various parts of XFS and iomap so that metadata health failures, file I/O errors, and major changes in filesystem state (unmounts, shutdowns, etc.) can be observed by programs.
When an event occurs, the hook functions queue an event object to each event anonfd for later processing. Programs must have CAP_SYS_ADMIN to open the anonfd and there's a maximum event lag to prevent resource overconsumption. The events themselves can be read() from the anonfd as C structs for the xfs_healer daemon.
In userspace, we create a new daemon program that will read the event objects and initiate repairs automatically. This daemon is managed entirely by systemd and will not block unmounting of the filesystem unless repairs are ongoing. They are auto-started by a starter service that uses fanotify.
v9: move listmount/statmount to libfrog; improve documentation about why
we dance with getmntent; enhance getmntent reconnection with listmount; move --svcname helpers to xfs_{healer,scrub}; improve commit messages; various tweaks to fstests
v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=health-monitoring
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=health-monitoring
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=health-monitoring
---
Commits in this patchset:
* libfrog: add a function to grab the path from an open fd and a file handle
* libfrog: create healthmon event log library functions
* libfrog: add support code for starting systemd services programmatically
* libfrog: hoist a couple of service helper functions
* libfrog: add wrappers for listmount and statmount
* man2: document the healthmon ioctl
* man2: document the media verification ioctl
* xfs_io: monitor filesystem health events
* xfs_io: add a media verify command
* xfs_healer: create daemon to listen for health events
* xfs_healer: enable repairing filesystems
* xfs_healer: use getparents to look up file names
* xfs_healer: create a per-mount background monitoring service
* xfs_healer: create a service to start the per-mount healer service
* xfs_healer: don't start service if kernel support unavailable
* xfs_healer: use the autofsck fsproperty to select mode
* xfs_healer: run full scrub after lost corruption events or targeted repair failure
* xfs_healer: use getmntent to find moved filesystems
* xfs_healer: use statmount to find moved filesystems even faster
* xfs_healer: validate that repair fds point to the monitored fs
* xfs_healer: add a manual page
* xfs_scrub: use the verify media ioctl during phase 6 if possible
* xfs_scrub: perform media scanning of the log region
* xfs_scrub: print systemd service names
* xfs_io: add listmount and statmount commands
* mkfs: enable online repair if all backrefs are enabled
* debian/control: listify the build dependencies
* debian: enable xfs_healer on the root filesystem by default
--- healer/xfs_healer.h | 92 +++ include/linux.h | 8 io/io.h | 8 libfrog/flagmap.h | 23 + libfrog/fsproperties.h | 5 libfrog/getparents.h | 4 libfrog/healthevent.h | 55 ++ libfrog/statmount.h | 104 ++++ libfrog/systemd.h | 55 ++ scrub/disk.h | 11 scrub/xfs_scrub.h | 3 Makefile | 5 configure.ac | 13 debian/control | 14 - debian/postinst | 8 debian/prerm | 13 debian/rules | 3 healer/Makefile | 69 ++ healer/fsrepair.c | 342 ++++++++++++ healer/system-xfs_healer.slice | 31 + healer/weakhandle.c | 296 +++++++++++ healer/xfs_healer.c | 666 ++++++++++++++++++++++++ healer/[email protected] | 108 ++++ healer/xfs_healer_start.c | 368 +++++++++++++ healer/xfs_healer_start.service.in | 85 +++ include/builddefs.in | 15 + io/Makefile | 9 io/healthmon.c | 186 +++++++ io/init.c | 3 io/listmount.c | 361 +++++++++++++ io/verify_media.c | 180 ++++++ libfrog/Makefile | 19 + libfrog/flagmap.c | 79 +++ libfrog/getparents.c | 93 +++ libfrog/healthevent.c | 477 +++++++++++++++++ libfrog/statmount.c | 76 +++ libfrog/systemd.c | 181 +++++++ m4/package_libcdev.m4 | 129 +++++ man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 | 75 +++ man/man2/ioctl_xfs_health_monitor.2 | 464 +++++++++++++++++ man/man2/ioctl_xfs_verify_media.2 | 185 +++++++ man/man8/Makefile | 40 + man/man8/xfs_healer.8 | 109 ++++ man/man8/xfs_healer_start.8 | 37 + man/man8/xfs_io.8 | 133 +++++ mkfs/xfs_mkfs.c | 9 scrub/Makefile | 14 - scrub/disk.c | 38 + scrub/phase1.c | 25 + scrub/phase6.c | 11 scrub/read_verify.c | 2 scrub/xfs_scrub.c | 82 ++- 52 files changed, 5357 insertions(+), 64 deletions(-) create mode 100644 healer/xfs_healer.h create mode 100644 libfrog/flagmap.h create mode 100644 libfrog/healthevent.h create mode 100644 libfrog/statmount.h create mode 100644 libfrog/systemd.h create mode 100644 debian/prerm create mode 100644 healer/Makefile create mode 100644 healer/fsrepair.c create mode 100644 healer/system-xfs_healer.slice create mode 100644 healer/weakhandle.c create mode 100644 healer/xfs_healer.c create mode 100644 healer/[email protected] create mode 100644 healer/xfs_healer_start.c create mode 100644 healer/xfs_healer_start.service.in create mode 100644 io/healthmon.c create mode 100644 io/listmount.c create mode 100644 io/verify_media.c create mode 100644 libfrog/flagmap.c create mode 100644 libfrog/healthevent.c create mode 100644 libfrog/statmount.c create mode 100644 libfrog/systemd.c create mode 100644 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 create mode 100644 man/man2/ioctl_xfs_health_monitor.2 create mode 100644 man/man2/ioctl_xfs_verify_media.2 create mode 100644 man/man8/xfs_healer.8 create mode 100644 man/man8/xfs_healer_start.8
PATCH 1/28 libfrog: add a function to grab the path from an open fd and a file handle
From: Darrick J. Wong <[email protected]>
handle_walk_paths operates on a file handle, but requires that the fs has been registered with libhandle via path_to_fshandle. For a normal libhandle client this is the desirable behavior because the application *should* maintain an open fd to the filesystem mount.
However for xfs_healer this isn't going to work well because the healer mustn't pin the mount while it's running. It's smart enough to know how to find and reconnect to the mountpoint, but libhandle doesn't have any such concept.
Therefore, alter the libfrog getparents code so that xfs_healer can pass in the mountpoint and reconnected fd without needing libhandle. All we're really doing here is trying to obtain a user-visible path for a file that encountered problems for logging purposes; if it fails, we'll fall back to logging the inode number.
--- libfrog/getparents.h | 4 ++ libfrog/getparents.c | 93 ++++++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 82 insertions(+), 15 deletions(-)
libfrog/getparents.h +4 -0
@@ -39,4 +39,8 @@ int fd_to_path(int fd, size_t ioctl_bufsize, char *path, size_t pathlen);int handle_to_path (const void *hanp,size_t hlen,size_t ioctl_bufsize,char *path,size_t pathlen); +int handle_walk_paths_fd (const char *mntpt,int mntfd,const void *hanp, +size_t hanlen,size_t ioctl_bufsize,walk_path_fn fn, +void *arg); +#endif /* __LIBFROG_GETPARENTS_H_ */
libfrog/getparents.c +78 -15
141 lines not on this page · raw message
PATCH 2/28 libfrog: create healthmon event log library functions
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add some helper functions to log health monitoring events so that xfs_io and xfs_healer can share logging code.
--- libfrog/flagmap.h | 20 +++ libfrog/healthevent.h | 43 ++++++ libfrog/Makefile | 4 + libfrog/flagmap.c | 62 ++++++++ libfrog/healthevent.c | 360 +++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 489 insertions(+) create mode 100644 libfrog/flagmap.h create mode 100644 libfrog/healthevent.h create mode 100644 libfrog/flagmap.c create mode 100644 libfrog/healthevent.c
libfrog/flagmap.h new file +20 -0
@@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025-2026 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef LIBFROG_FLAGMAP_H_ +#define LIBFROG_FLAGMAP_H_ + +struct flag_map { +unsigned long long flag; +const char *string; +}; + +void mask_to_string (const struct flag_map *map,unsigned long long mask, +const char *delimiter,char *buf,size_t bufsize); + +const char *value_to_string (const struct flag_map *map, +unsigned long long value); + +#endif /* LIBFROG_FLAGMAP_H_ */
libfrog/healthevent.h new file +43 -0
43 lines not on this page · raw message
libfrog/Makefile +4 -0
@@ -19,11 +19,13 @@ bulkstat.c \ convert.c \ crc32.c \ file_exchange.c \ +flagmap.c \ fsgeom.c \ fsproperties.c \ fsprops.c \ getparents.c \ histogram.c \ +healthevent.c \ file_attr.c \ list_sort.c \ linux.c \
@@ -51,11 +53,13 @@ dahashselftest.h \ div64.h \ fakelibattr.h \ file_exchange.h \ +flagmap.h \ fsgeom.h \ fsproperties.h \ fsprops.h \ getparents.h \ handle_priv.h \ +healthevent.h \ histogram.h \ file_attr.h \ logging.h \
libfrog/flagmap.c new file +62 -0
62 lines not on this page · raw message
libfrog/healthevent.c new file +360 -0
360 lines not on this page · raw message
PATCH 3/28 libfrog: add support code for starting systemd services programmatically
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add some simple routines for computing the name of systemd service instances and starting systemd services. These will be used by the xfs_healer_start service to start per-filesystem xfs_healer service instances.
Note that we run systemd helper programs as subprocesses for a couple of reasons. First, the path-escaping functionality is not a part of any library-accessible API, which means it can only be accessed via systemd-escape(1). Second, although the service startup functionality can be reached via dbus, doing so would introduce a new library dependency. Systemd is also undergoing a dbus -> varlink RPC transition so we avoid that mess by calling the cli systemctl(1) program.
--- libfrog/systemd.h | 20 +++++ configure.ac | 1 include/builddefs.in | 1 libfrog/Makefile | 6 ++ libfrog/systemd.c | 181 +++++++++++++++++++++++++++++++++++++++++++++++++ m4/package_libcdev.m4 | 19 +++++ 6 files changed, 228 insertions(+) create mode 100644 libfrog/systemd.h create mode 100644 libfrog/systemd.c
libfrog/systemd.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (c) 2026 Oracle. All rights reserved. +* All Rights Reserved. +*/ +#ifndef __LIBFROG_SYSTEMD_H__ +#define __LIBFROG_SYSTEMD_H__ + +int systemd_path_instance_unit_name (const char *unit_template, +const char *path,char *unitname,size_t unitnamelen); + +enum systemd_unit_manage { +UM_STOP , +UM_START , +UM_RESTART , +}; + +int systemd_manage_unit (enum systemd_unit_manage how,const char *unitname); + +#endif /* __LIBFROG_SYSTEMD_H__ */
configure.ac +1 -0
@@ -182,6 +182,7 @@ AC_CONFIG_UDEV_RULE_DIR AC_HAVE_BLKID_TOPO AC_HAVE_TRIVIAL_AUTO_VAR_INIT AC_STRERROR_R_RETURNS_STRING +AC_HAVE_CLOSE_RANGE if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then AC_PACKAGE_CHECK_UBSAN
include/builddefs.in +1 -0
@@ -118,6 +118,7 @@ HAVE_UDEV = @have_udev@ UDEV_RULE_DIR = @udev_rule_dir@ HAVE_LIBURCU_ATOMIC64 = @have_liburcu_atomic64@ STRERROR_R_RETURNS_STRING = @strerror_r_returns_string@ +HAVE_CLOSE_RANGE = @have_close_range@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall # -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
libfrog/Makefile +6 -0
@@ -36,6 +36,7 @@ ptvar.c \ radix-tree.c \ randbytes.c \ scrub.c \ +systemd.c \ util.c \ workqueue.c \ zones.c
@@ -70,6 +71,7 @@ radix-tree.h \ randbytes.h \ scrub.h \ statx.h \ +systemd.h \ workqueue.h \ zones.h
@@ -90,6 +92,10 @@ ifeq ($(HAVE_GETRANDOM_NONBLOCK),yes) LCFLAGS += -DHAVE_GETRANDOM_NONBLOCK endif +ifeq ($(HAVE_CLOSE_RANGE),yes) +CFLAGS += -DHAVE_CLOSE_RANGE +endif + default: ltdepend $(LTLIBRARY) $(GETTEXT_PY) crc32table.h: gen_crc32table.c crc32defs.h
libfrog/systemd.c new file +181 -0
181 lines not on this page · raw message
m4/package_libcdev.m4 +19 -0
@@ -347,3 +347,22 @@ puts(strerror_r(0, buf, sizeof(buf))); CFLAGS="$OLD_CFLAGS" AC_SUBST(strerror_r_returns_string) ]) + +# +# Check if close_range exists +# +AC_DEFUN([AC_HAVE_CLOSE_RANGE], + [AC_MSG_CHECKING([for close_range]) + AC_LINK_IFELSE( + [AC_LANG_PROGRAM([[ +#define _GNU_SOURCE +#include <unistd.h> +#include <linux/close_range.h> + ]], [[ +close_range(0, 0, 0); + ]]) + ], have_close_range=yes + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no)) + AC_SUBST(have_close_range) + ])
Re: [PATCH 03/28] libfrog: add support code for starting systemd services programmatically
Quoted 7 lines
libfrog/systemd.c new file +7
-0+static void +close_fds (void ) +{ +int max_fd =sysconf (_SC_OPEN_MAX); +int fd; +#ifdef HAVE_CLOSE_RANGE +int ret;
libfrog/systemd.c line 31
Quoted 10 lines
libfrog/systemd.c new file +10
-0+#endif + +if (max_fd <1 ) + max_fd =1024 ; + +#ifdef HAVE_CLOSE_RANGE + ret =close_range (STDERR_FILENO +1 , max_fd,0 ); +if (!ret) +return ; +#endif
ret is only used to check that it is zero here, so maybe do away with it and avoid the extra ifdef?
Otherwise looks good:
Re: [PATCH 03/28] libfrog: add support code for starting systemd services programmatically
Quoted 12 lines · Christoph Hellwig
Quoted 12 lines
libfrog/systemd.c new file +12
-0+static void +close_fds (void ) +{ +int max_fd =sysconf (_SC_OPEN_MAX); +int fd; +#ifdef HAVE_CLOSE_RANGE +int ret; +#endif + +if (max_fd <1 ) + max_fd =1024 ; +
Quoted 8 lines
libfrog/systemd.c line 31
Quoted 5 lines
libfrog/systemd.c new file +5
-0+#ifdef HAVE_CLOSE_RANGE + ret =close_range (STDERR_FILENO +1 , max_fd,0 ); +if (!ret) +return ; +#endif ret is only used to check that it is zero here, so maybe do away with it and avoid the extra ifdef?
Fixed, thanks for the feedback!
--D
Quoted 4 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
PATCH 4/28 libfrog: hoist a couple of service helper functions
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Hoist a couple of service/daemon-related helper functions to libfrog so that we can share the code between xfs_scrub and xfs_healer.
--- libfrog/systemd.h | 28 ++++++++++++++++++++++++++++ scrub/xfs_scrub.c | 32 +++++++++----------------------- 2 files changed, 37 insertions(+), 23 deletions(-)
libfrog/systemd.h +28 -0
@@ -17,4 +17,32 @@ enum systemd_unit_manage {int systemd_manage_unit (enum systemd_unit_manage how,const char *unitname); +static inline bool systemd_is_service (void ) +{ +return getenv ("SERVICE_MODE" ) !=NULL ; +} + +/* Special processing for a service/daemon program that is exiting. */ +static inline int +systemd_service_exit (int ret) +{ +/* +* We have to sleep 2 seconds here because journald uses the pid to +* connect our log messages to the systemd service. This is critical +* for capturing all the log messages if the service fails, because +* failure analysis tools use the service name to gather log messages +* for reporting. +*/ +sleep (2 ); + +/* +* If we're being run as a service, the return code must fit the LSB +* init script action error guidelines, which is to say that we +* compress all errors to 1 ("generic or unspecified error", LSB 5.0 +* section 22.2) and hope the admin will scan the log for what actually +* happened. +*/ +return ret !=0 ?EXIT_FAILURE :EXIT_SUCCESS ; +} +#endif /* __LIBFROG_SYSTEMD_H__ */
scrub/xfs_scrub.c +9 -23
@@ -19,6 +19,7 @@#include "unicrash.h" #include "progress.h" #include "libfrog/histogram.h" +#include "libfrog/systemd.h" /* * XFS Online Metadata Scrub (and Repair)
@@ -866,8 +867,7 @@ main(if (stdout_isatty && !progress_fp) progress_fp =fdopen (1 ,"w+" ); -if (getenv ("SERVICE_MODE" )) - is_service = true; + is_service =systemd_is_service ();/* Initialize overall phase stats. */ error =phase_start (&all_pi,0 ,NULL );
@@ -960,29 +960,15 @@ main(hist_free (&ctx.datadev_hist);hist_free (&ctx.rtdev_hist); -/* -* If we're being run as a service, the return code must fit the LSB -* init script action error guidelines, which is to say that we -* compress all errors to 1 ("generic or unspecified error", LSB 5.0 -* section 22.2) and hope the admin will scan the log for what -* actually happened. -* -* We have to sleep 2 seconds here because journald uses the pid to -* connect our log messages to the systemd service. This is critical -* for capturing all the log messages if the scrub fails, because the -* fail service uses the service name to gather log messages for the -* error report. -* -* Note: We don't count a lack of kernel support as a service failure -* because we haven't determined that there's anything wrong with the -* filesystem. -*/ if (is_service) { -sleep (2 ); +/* +* Note: We don't count a lack of kernel support as a service +* failure because we haven't determined that there's anything +* wrong with the filesystem. +*/ if (!ctx.scrub_setup_succeeded) -return 0 ; -if (ret !=SCRUB_RET_SUCCESS ) -return 1 ; + ret =0 ; +return systemd_service_exit (ret); }return ret;
PATCH 5/28 libfrog: add wrappers for listmount and statmount
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add some wrappers for listmount and statmount so that we don't have to open-code the kernel ABI quirks in every utility program that uses it. Note that glibc seems to have discussed providing a wrapper in late 2023 but took no action; and the listmount manpage says that there is no glibc wrapper.
--- include/linux.h | 8 +++- libfrog/statmount.h | 104 +++++++++++++++++++++++++++++++++++++++++++++++++ configure.ac | 5 ++ include/builddefs.in | 7 +++ libfrog/Makefile | 9 ++++ libfrog/statmount.c | 76 ++++++++++++++++++++++++++++++++++++ m4/package_libcdev.m4 | 86 +++++++++++++++++++++++++++++++++++++++++ 7 files changed, 294 insertions(+), 1 deletion(-) create mode 100644 libfrog/statmount.h create mode 100644 libfrog/statmount.c
include/linux.h +7 -1
@@ -32,7 +32,13 @@#ifdef OVERRIDE_SYSTEM_FSXATTR # define fsxattr sys_fsxattr#endif -#include <linux/fs.h> /* fsxattr defintion for new kernels */ +#ifdef OVERRIDE_SYSTEM_STATMOUNT +# define statmount sys_statmount +#endif +#include <linux/fs.h> /* fsxattr/statmount defintion for new kernels */ +#ifdef OVERRIDE_SYSTEM_STATMOUNT +# undef statmount +#endif #ifdef OVERRIDE_SYSTEM_FSXATTR # undef fsxattr#endif
libfrog/statmount.h new file +104 -0
104 lines not on this page · raw message
configure.ac +5 -0
@@ -183,6 +183,11 @@ AC_HAVE_BLKID_TOPO AC_HAVE_TRIVIAL_AUTO_VAR_INIT AC_STRERROR_R_RETURNS_STRING AC_HAVE_CLOSE_RANGE +AC_HAVE_LISTMOUNT +if test "$have_listmount" = "yes"; then + AC_HAVE_LISTMOUNT_NS_FD + AC_HAVE_STATMOUNT_SUPPORTED_MASK +fi if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then AC_PACKAGE_CHECK_UBSAN
include/builddefs.in +7 -0
@@ -119,6 +119,10 @@ UDEV_RULE_DIR = @udev_rule_dir@ HAVE_LIBURCU_ATOMIC64 = @have_liburcu_atomic64@ STRERROR_R_RETURNS_STRING = @strerror_r_returns_string@ HAVE_CLOSE_RANGE = @have_close_range@ +HAVE_LISTMOUNT = @have_listmount@ +HAVE_LISTMOUNT_NS_FD = @have_listmount_ns_fd@ +HAVE_STATMOUNT_SUPPORTED_MASK = @have_statmount_supported_mask@ +NEED_INTERNAL_STATMOUNT = @need_internal_statmount@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall # -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
@@ -141,6 +145,9 @@ endif ifeq ($(NEED_INTERNAL_STATX),yes) PCFLAGS+= -DOVERRIDE_SYSTEM_STATX endif +ifeq ($(NEED_INTERNAL_STATMOUNT),yes) +PCFLAGS+= -DOVERRIDE_SYSTEM_STATMOUNT +endif ifeq ($(HAVE_GETFSMAP),yes) PCFLAGS+= -DHAVE_GETFSMAP endif
libfrog/Makefile +9 -0
@@ -96,6 +96,15 @@ ifeq ($(HAVE_CLOSE_RANGE),yes) CFLAGS += -DHAVE_CLOSE_RANGE endif +ifeq ($(HAVE_LISTMOUNT),yes) +CFILES += statmount.c +HFILES += statmount.h +endif + +ifeq ($(HAVE_LISTMOUNT_NS_FD),yes) +CFLAGS+=-DHAVE_LISTMOUNT_NS_FD +endif + default: ltdepend $(LTLIBRARY) $(GETTEXT_PY) crc32table.h: gen_crc32table.c crc32defs.h
libfrog/statmount.c new file +76 -0
76 lines not on this page · raw message
m4/package_libcdev.m4 +86 -0
89 lines not on this page · raw message
Re: [PATCH 05/28] libfrog: add wrappers for listmount and statmount
Looks good:
PATCH 6/28 man2: document the healthmon ioctl
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Document the XFS_IOC_HEALTH_MONITOR andXFS_IOC_HEALTH_FD_ON_MONITORED_FS ioctls.
--- man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 | 75 ++++ man/man2/ioctl_xfs_health_monitor.2 | 464 ++++++++++++++++++++++++ 2 files changed, 539 insertions(+) create mode 100644 man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 create mode 100644 man/man2/ioctl_xfs_health_monitor.2
man/man2/ioctl_xfs_health_fd_on_monitored_fs.2 new file +75 -0
@@ -0,0 +1,75 @@ +.\" Copyright (c) 2025-2026, Oracle. All rights reserved. +.\" +.\" %%%LICENSE_START(GPLv2+_DOC_FULL) +.\" SPDX-License-Identifier: GPL-2.0+ +.\" %%%LICENSE_END +.TH IOCTL-XFS-HEALTH-FD-ON-MONITORED-FS 2 2026-01-04 "XFS" +.SH NAME +ioctl_xfs_health_fd_on_monitored_fs \- check if the given fd belongs to the same fs being monitored +.SH SYNOPSIS +.br +.B #include <xfs/xfs_fs.h> +.PP +.BI "int ioctl(int " healthmon_fd ", XFS_IOC_HEALTH_FD_ON_MONITORED_FS, struct xfs_health_file_on_monitored_fs *" arg ); +.SH DESCRIPTION +This XFS healthmon fd ioctl asks the kernel driver if the file descriptor +passed in via +.I arg +points to a file on the same filesystem that is being monitored by +.IR healthmon_fd . +The file descriptor is conveyed in a structure of the following form: +.PP +.in +4n +.nf +struct xfs_health_file_on_monitored_fs { + __s32 fd; + __u32 flags; +}; +.fi +.in +.PP +The field +.I flags +must be zero. +.PP +The field +.I fd +is a descriptor of an open file. +.PP +The argument +.I healthmon_fd +must be a file opened via the +.B XFS_IOC_HEALTH_MONITOR +ioctl. +.SH RETURN VALUE +On error, \-1 is returned, and +.I errno +is set to indicate the error. +If the file descriptor points to a file on the same filesystem that is being +monitored, 0 is returned. +.PP +.SH ERRORS +Error codes can be one of, but are not limited to, the following: +.TP +.B ESTALE +The open file is not on the same filesystem that is being monitored. +.TP +.B EINVAL +One or more of the arguments specified is invalid. +.TP +.B EBADF +.I arg.fd +does not refer to an open file. +.TP +.B EFAULT +The +.I arg +structure could not be copied into the kernel. +.TP +.B ENOTTY +.I healthmon_fd +is not a XFS health monitoring file. +.SH CONFORMING TO +This API is specific to XFS filesystem on the Linux kernel. +.SH SEE ALSO +.BR ioctl_xfs_health_monitor (2)
man/man2/ioctl_xfs_health_monitor.2 new file +464 -0
464 lines not on this page · raw message
PATCH 7/28 man2: document the media verification ioctl
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Document XFS_IOC_VERIFY_MEDIA, which is a new ioctl for xfs_scrub to perform media scans on the disks underneath the filesystem. This will enable media errors to be reported to xfs_healer and fsnotify.
--- man/man2/ioctl_xfs_verify_media.2 | 185 +++++++++++++++++++++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 man/man2/ioctl_xfs_verify_media.2
man/man2/ioctl_xfs_verify_media.2 new file +185 -0
@@ -0,0 +1,185 @@ +.\" Copyright (c) 2025-2026, Oracle. All rights reserved. +.\" +.\" %%%LICENSE_START(GPLv2+_DOC_FULL) +.\" SPDX-License-Identifier: GPL-2.0+ +.\" %%%LICENSE_END +.TH IOCTL-XFS-VERIFY-MEDIA 2 2026-01-09 "XFS" +.SH NAME +ioctl_xfs_verify_media \- verify the media of the devices backing XFS +.SH SYNOPSIS +.br +.B #include <xfs/xfs_fs.h> +.PP +.BI "int ioctl(int " fd ", XFS_IOC_VERIFY_MEDIA, struct xfs_verify_media *" arg ); +.SH DESCRIPTION +Verify the media of a storage device backing an XFS filesystem. +If errors are found, report the error to the kernel so that it can generate +health events for the health monitoring system and fsnotify. +The verification request is conveyed in a structure of the following form: +.PP +.in +4n +.nf +struct xfs_verify_error { + __u32 me_dev; + __u32 me_flags; + __u64 me_start_daddr; + __u64 me_end_daddr; + __u32 me_ioerror; + __u32 me_pad; +}; +.fi +.in +.PP +The field +.I me_pad +must be zero. +.PP +The field +.I me_ioerror +will be set if the ioctl returns success. +.PP +The fields +.I me_start_daddr +and +.I me_end_daddr +are the range of the storage device to verify. +Both values must be in units of 512-byte blocks. +The +.I me_start_daddr +field is inclusive, and the +.I me_end_daddr +field is exclusive. +If +.I me_end_daddr +is larger than the size of the device, the kernel will set it to the size of +the device. + +If the system call returns success and any part of the storage device range was +successfully verified, the +.I me_start_daddr +field will be updated to reflect the successful verification. +If after this update the +.I me_start_daddr +is equal to +.IR me_end_daddr , +then the entire range was verified successfully. + +If not, then a media error was encountered and the caller should generate a +series of secondary calls to this ioctl with smaller ranges to discover the +exact location and type of media error. +The type of media error will be written to the +.I me_ioerror +field. + +.PP +The field +.I me_dev +must be one of the following values: +.RS 0.4i +.TP +.B XFS_DEV_DATA +Verify the data device. +.TP +.B XFS_DEV_LOG +Verify the external log device. +.TP +.B XFS_DEV_RT +Verify the realtime device. +.RE +.PP +The field +.I me_flags +is a bitmask of one of the following values: +.RS 0.4i +.TP +.B XFS_VERIFY_MEDIA_REPORT +Report all media errors to fsnotify. +.RE + +The +.IR me_max_io_size +field, if nonzero, will be used as advice for the maximum size of the IO to +send to the device. + +The +.I me_rest_us +field will cause the kernel to pause for this many microseconds between IO +requests. + +.SH RETURN VALUE +On runtime error, \-1 is returned, and +.I errno +is set to indicate the error. +If 0 is returned, then +.I start_daddr +or +.I ioerror +will be updated. +.PP +.SH ERRORS +Error codes can be one of, but are not limited to, the following: +.TP +.B EPERM +The calling process does not have sufficient privilege. +.TP +.B EINVAL +One or more of the arguments specified is invalid. +.TP +.B EFAULT +The +.I arg +structure could not be copied into the kernel. +.TP +.B ENODEV +The device is not present. +.TP +.B ENOMEM +There was not enough memory to perform the verification. + +.SH I/O ERRORS +The +.I ioerror +field could be set to one of the following: +.TP +.B 0 +The verification I/O succeeded. +.TP +.B EOPNOTSUPP +.TP +.B ETIMEDOUT +The kernel timed out the verification I/O command. +.TP +.B ENOLINK +The transportation link to the storage device was down temporarily. +.TP +.B EREMOTEIO +The storage target controller suffered a critical error. +.TP +.B ENODATA +The storage target media suffered a critical error. +.TP +.B EILSEQ +Storage protection metadata did not validate successfully. +.TP +.B ENOMEM +There was not enough memory to allocate an I/O request. +.TP +.B ENODEV +The storage device is offline. +.TP +.B ETIME +The storage device timed out the I/O command. +.TP +.B EINVAL +The I/O request was rejected by the device for being invalid. +.TP +.B EIO +An I/O error occurred but no specific details are available. +.RE +.PP +This list is not exhaustive and may grow in the future. + +.SH CONFORMING TO +This API is specific to XFS filesystem on the Linux kernel. +.SH SEE ALSO +.BR ioctl_xfs_health_monitor (2)
PATCH 8/28 xfs_io: monitor filesystem health events
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a subcommand to monitor for health events generated by the kernel.
--- io/io.h | 1 io/Makefile | 1 io/healthmon.c | 186 +++++++++++++++++++++++++++++++++++++++++++++++++++++ io/init.c | 1 man/man8/xfs_io.8 | 25 +++++++ 5 files changed, 214 insertions(+) create mode 100644 io/healthmon.c
io/io.h +1 -0
@@ -162,3 +162,4 @@ extern void bulkstat_init(void);void exchangerange_init (void );void fsprops_init (void );void aginfo_init (void ); +void healthmon_init (void );
io/Makefile +1 -0
@@ -25,6 +25,7 @@ CFILES = \ fsuuid.c \ fsync.c \ getrusage.c \ + healthmon.c \ imap.c \ init.c \ inject.c \
io/healthmon.c new file +186 -0
186 lines not on this page · raw message
io/init.c +1 -0
@@ -92,6 +92,7 @@ init_commands(void)crc32cselftest_init ();exchangerange_init ();fsprops_init (); +healthmon_init (); }/*
man/man8/xfs_io.8 +25 -0
@@ -1356,6 +1356,31 @@ .SH FILESYSTEM COMMANDS .B thaw Undo the effects of a filesystem freeze operation. Only available in expert mode and requires privileges. +.TP +.BI "healthmon [ \-c " bufsize " ] [ \-c ] [ \-d " delay_ms " ] [ \-p ] [ \-v ]" +Watch for filesystem health events and write them to the console. +.RE +.RS 1.0i +.PD 0 +.TP +.BI "\-b " bufsize +Use a buffer of this size to read events from the kernel. +.TP +.BI \-c +Close the open file and replace it with the monitor file. +.TP +.BI "\-d " delay_ms +Sleep for this long between read attempts. +.TP +.B \-p +Probe for the existence of the functionality by opening the monitoring fd and +closing it immediately. +.TP +.BI \-v +Request all health events, even if nothing changed. +.PD +.RE + .TP .BI "inject [ " tag " ]" Inject errors into a filesystem to observe filesystem behavior at
PATCH 9/28 xfs_io: add a media verify command
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a subcommand to invoke the media verification ioctl to make sure that we can actually check the storage underneath an xfs filesystem.
--- io/io.h | 1 io/Makefile | 3 + io/init.c | 1 io/verify_media.c | 180 +++++++++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_io.8 | 42 ++++++++++++ 5 files changed, 226 insertions(+), 1 deletion(-) create mode 100644 io/verify_media.c
io/io.h +1 -0
@@ -163,3 +163,4 @@ void exchangerange_init(void);void fsprops_init (void );void aginfo_init (void );void healthmon_init (void ); +void verifymedia_init (void );
io/Makefile +2 -1
@@ -51,7 +51,8 @@ CFILES = \ sync.c \ sync_file_range.c \ truncate.c \ - utimes.c + utimes.c \ + verify_media.c LLDLIBS = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG) $(LIBPTHREAD) $(LIBUUID) LTDEPENDENCIES = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG)
io/init.c +1 -0
@@ -93,6 +93,7 @@ init_commands(void)exchangerange_init ();fsprops_init ();healthmon_init (); +verifymedia_init (); }/*
io/verify_media.c new file +180 -0
180 lines not on this page · raw message
man/man8/xfs_io.8 +42 -0
48 lines not on this page · raw message
PATCH 10/28 xfs_healer: create daemon to listen for health events
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a daemon program that can listen for and log health events. Eventually this will be used to self-heal filesystems in real time.
Because events can take a while to process, the main thread reads event objects from the healthmon fd and dispatches them to a background workqueue as quickly as it can. This split of responsibilities is necessary because the kernel event queue will drop events if the queue fills up, and each event can take some time to process (logging, repairs, etc.) so we don't want to lose events.
To be clear, xfs_healer and xfs_scrub are complementary tools:
Scrub walks the whole filesystem, finds stuff that needs fixing or rebuilding, and rebuilds it. This is sort of analogous to a patrol scrub.
Healer listens for metadata corruption messages from the kernel and issues a targeted repair of that structure. This is kind of like an ondemand scrub.
My end goal is that xfs_healer (the service) is active all the time and can respond instantly to a corruption report, whereas xfs_scrub (the service) gets run periodically as a cron job.
xfs_healer can decide that it's overwhelmed with problems and start xfs_scrub to deal with the mess. Ideally you don't crash the filesystem and then have to use xfs_repair to smash your way back to a mountable filesystem.
By default we run xfs_healer as a background service, which means that we only start two threads -- one to read the events, and another to process them. In other words, we try not to use all available hardware resources for repairs. The foreground mode switch starts up a large number of threads to try to increase parallelism, which may or may not be useful for repairs depending on how much metadata the kernel needs to scan.
--- healer/xfs_healer.h | 47 ++++++ Makefile | 5 + configure.ac | 6 + healer/Makefile | 35 ++++ healer/xfs_healer.c | 391 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/builddefs.in | 1 6 files changed, 485 insertions(+) create mode 100644 healer/xfs_healer.h create mode 100644 healer/Makefile create mode 100644 healer/xfs_healer.c
healer/xfs_healer.h new file +47 -0
@@ -0,0 +1,47 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025-2026 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef XFS_HEALER_XFS_HEALER_H_ +#define XFS_HEALER_XFS_HEALER_H_ + +extern char *progname; + +/* +* When running in environments with restrictive security policies, healer +* might not be allowed to access the global mount tree. However, processes +* are usually still allowed to see their own mount tree, so use this path for +* all mount table queries. +*/ +#define _PATH_PROC_MOUNTS "/proc/self/mounts" + +struct healer_ctx { +/* CLI options, must be int */ +int debug; +int log; +int everything; +int foreground; + +/* fd and fs geometry for mount */ +struct xfs_fd mnt; + +/* Shared reference to the user's mountpoint for logging */ +const char *mntpoint; + +/* Shared reference to the getmntent fsname for reconnecting */ +const char *fsname; + +/* file stream of monitor and buffer */ +FILE *mon_fp; +char *mon_buf; + +/* coordinates logging printfs */ +pthread_mutex_t conlock; + +/* event queue */ +struct workqueue event_queue; +bool queue_active; +}; + +#endif /* XFS_HEALER_XFS_HEALER_H_ */
Makefile +5 -0
@@ -69,6 +69,10 @@ ifeq ("$(ENABLE_SCRUB)","yes") TOOL_SUBDIRS += scrub endif +ifeq ("$(ENABLE_HEALER)","yes") +TOOL_SUBDIRS += healer +endif + ifneq ("$(XGETTEXT)","") TOOL_SUBDIRS += po endif
@@ -100,6 +104,7 @@ mkfs: libxcmd spaceman: libxcmd libhandle scrub: libhandle libxcmd rtcp: libfrog +healer: libhandle ifeq ($(HAVE_BUILDDEFS), yes) include $(BUILDRULES)
configure.ac +6 -0
@@ -110,6 +110,12 @@ AC_ARG_ENABLE(libicu, [ --enable-libicu=[yes/no] Enable Unicode name scanning in xfs_scrub (libicu) [default=probe]],, enable_libicu=probe) +# Enable xfs_healer build +AC_ARG_ENABLE(healer, +[ --enable-healer=[yes/no] Enable build of xfs_healer utility [[default=yes]]],, + enable_healer=yes) +AC_SUBST(enable_healer) + # # If the user specified a libdir ending in lib64 do not append another # 64 to the library names.
healer/Makefile new file +35 -0
@@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2024-2026 Oracle. All Rights Reserved. +# + +TOPDIR = .. +builddefs=$(TOPDIR)/include/builddefs +include $(builddefs) + +INSTALL_HEALER = install-healer + +LTCOMMAND = xfs_healer + +CFILES = \ +xfs_healer.c + +HFILES = \ +xfs_healer.h + +LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD) +LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG) +LLDFLAGS = -static + +default: depend $(LTCOMMAND) + +include $(BUILDRULES) + +install: $(INSTALL_HEALER) + +install-healer: default + $(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR) + $(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR) + +install-dev: + +-include .dep
healer/xfs_healer.c new file +391 -0
391 lines not on this page · raw message
include/builddefs.in +1 -0
@@ -91,6 +91,7 @@ ENABLE_SHARED = @enable_shared@ ENABLE_GETTEXT = @enable_gettext@ ENABLE_EDITLINE = @enable_editline@ ENABLE_SCRUB = @enable_scrub@ +ENABLE_HEALER = @enable_healer@ HAVE_ZIPPED_MANPAGES = @have_zipped_manpages@
PATCH 11/28 xfs_healer: enable repairing filesystems
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make it so that our health monitoring daemon can initiate repairs in response to reports of corrupt filesystem metadata. Repairs are initiated from the background workers as explained in the previous patch.
Note that just like xfs_scrub, xfs_healer's ability to repair metadata relies heavily on back references such as reverse mappings and directory parent pointers to add redundancy to the filesystem. Check for these two features and whine a bit if they are missing, just like scrub.
There's a bit of trickery with the fd that is used to initiate repairs in the kernel. Because an open fd will pin the filesystem in memory, xfs_healer can only hold an open fd to the target filesystem while it's performing repairs. Therefore, at startup xfs_healer must sample enough information about the target filesystem to reconnect to it later on. Currently, the fs source (aka the data device path) and the root directory handle are sufficient to do this.
Someday we might be able to have revocable fds, which would eliminate the need for such efforts in userspace.
--- healer/xfs_healer.h | 28 ++++++ libfrog/flagmap.h | 3 + libfrog/healthevent.h | 12 ++ healer/Makefile | 2 healer/fsrepair.c | 249 +++++++++++++++++++++++++++++++++++++++++++++++++ healer/weakhandle.c | 115 +++++++++++++++++++++++ healer/xfs_healer.c | 55 +++++++++++ libfrog/flagmap.c | 17 +++ libfrog/healthevent.c | 117 +++++++++++++++++++++++ 9 files changed, 598 insertions(+) create mode 100644 healer/fsrepair.c create mode 100644 healer/weakhandle.c
healer/xfs_healer.h +28 -0
@@ -8,6 +8,9 @@extern char *progname; +struct weakhandle ; +struct hme_prefix ; +/* * When running in environments with restrictive security policies, healer * might not be allowed to access the global mount tree. However, processes
@@ -22,6 +25,7 @@ struct healer_ctx {int log;int everything;int foreground; +int want_repair;/* fd and fs geometry for mount */ struct xfs_fd mnt;
@@ -32,6 +36,9 @@ struct healer_ctx {/* Shared reference to the getmntent fsname for reconnecting */ const char *fsname; +/* weak file handle so we can reattach to filesystem */ +struct weakhandle *wh; +/* file stream of monitor and buffer */ FILE *mon_fp;char *mon_buf;
@@ -44,4 +51,25 @@ struct healer_ctx {bool queue_active; }; +static inline bool healer_has_rmapbt (const struct healer_ctx *ctx) +{ +return ctx->mnt.fsgeom.flags &XFS_FSOP_GEOM_FLAGS_RMAPBT ; +} + +static inline bool healer_has_parent (const struct healer_ctx *ctx) +{ +return ctx->mnt.fsgeom.flags &XFS_FSOP_GEOM_FLAGS_PARENT ; +} + +/* repair.c */ +int repair_metadata (struct healer_ctx *ctx,const struct hme_prefix *pfx, +const struct xfs_health_monitor_event *hme); +bool healer_can_repair (struct healer_ctx *ctx); + +/* weakhandle.c */ +int weakhandle_alloc (int fd,const char *mountpoint,const char *fsname, +struct weakhandle **whp); +int weakhandle_reopen (struct weakhandle *wh,int *fd); +void weakhandle_free (struct weakhandle **whp); +#endif /* XFS_HEALER_XFS_HEALER_H_ */
libfrog/flagmap.h +3 -0
@@ -14,6 +14,9 @@ struct flag_map {void mask_to_string (const struct flag_map *map,unsigned long long mask,const char *delimiter,char *buf,size_t bufsize); +const char *lowest_set_mask_string (const struct flag_map *map, +unsigned long long mask); +const char *value_to_string (const struct flag_map *map,unsigned long long value);
libfrog/healthevent.h +12 -0
@@ -40,4 +40,16 @@ hme_prefix_init(void hme_report_event (const struct hme_prefix *pfx,const struct xfs_health_monitor_event *hme); +enum repair_outcome { +REPAIR_SUCCESS , +REPAIR_FAILED , +REPAIR_PROBABLY_OK , +REPAIR_UNNECESSARY , +}; + +void report_health_repair (const struct hme_prefix *pfx, +const struct xfs_health_monitor_event *hme, +uint32_t event_mask, +enum repair_outcome outcome); +#endif /* LIBFROG_HEALTHEVENT_H_ */
healer/Makefile +2 -0
@@ -11,6 +11,8 @@ INSTALL_HEALER = install-healer LTCOMMAND = xfs_healer CFILES = \ +fsrepair.c \ +weakhandle.c \ xfs_healer.c HFILES = \
healer/fsrepair.c new file +249 -0
249 lines not on this page · raw message
healer/weakhandle.c new file +115 -0
115 lines not on this page · raw message
healer/xfs_healer.c +55 -0
103 lines not on this page · raw message
libfrog/flagmap.c +17 -0
@@ -44,6 +44,23 @@ mask_to_string(snprintf (buf, bufsize,"%s0x%llx" , tag, mask & ~seen); } +/* +* Given a mapping of bits to strings and a bitmask, return the string +* corresponding to the lowest set bit in the mask. +*/ +const char * +lowest_set_mask_string ( +const struct flag_map *map, +unsigned long long mask) +{ +for (; map->string; map++) { +if (mask & map->flag) +return _ (map->string); + } + +return _ ("unknown flag" ); +} +/* * Given a mapping of values to strings and a value, return the matching string * or confusion.
libfrog/healthevent.c +117 -0
120 lines not on this page · raw message
PATCH 12/28 xfs_healer: use getparents to look up file names
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the kernel tells about something that happened to a file, use the GETPARENTS ioctl to try to look up the path to that file for more ergonomic reporting.
--- healer/xfs_healer.h | 6 ++++ healer/fsrepair.c | 16 ++++++++- healer/weakhandle.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++ healer/xfs_healer.c | 45 ++++++++++++++++++++++++++- 4 files changed, 149 insertions(+), 4 deletions(-)
healer/xfs_healer.h +6 -0
@@ -61,6 +61,10 @@ static inline bool healer_has_parent(const struct healer_ctx *ctx)return ctx->mnt.fsgeom.flags &XFS_FSOP_GEOM_FLAGS_PARENT ; } +void lookup_path (struct healer_ctx *ctx, +const struct xfs_health_monitor_event *hme, +struct hme_prefix *pfx); +/* repair.c */ int repair_metadata (struct healer_ctx *ctx,const struct hme_prefix *pfx,const struct xfs_health_monitor_event *hme);
@@ -71,5 +75,7 @@ int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname, struct weakhandle **whp);int weakhandle_reopen (struct weakhandle *wh,int *fd);void weakhandle_free (struct weakhandle **whp); +int weakhandle_getpath_for (struct weakhandle *wh,uint64_t ino,uint32_t gen, +char *path,size_t pathlen);#endif /* XFS_HEALER_XFS_HEALER_H_ */
healer/fsrepair.c +14 -2
@@ -164,7 +164,7 @@ try_repair_rtgroup(static void try_repair_inode (struct healer_ctx *ctx, -const struct hme_prefix *pfx, +const struct hme_prefix *orig_pfx,int mnt_fd,const struct xfs_health_monitor_event *hme) {
@@ -182,13 +182,25 @@ try_repair_inode( {0, 0}, };#undef X -const struct u32_scrub *f; +struct hme_prefix new_pfx; +const struct hme_prefix *pfx = orig_pfx; +const struct u32_scrub *f; foreach_scrub_type(f, hme->e.inode.mask, INODE_STRUCTURES) {enum repair_outcome outcome =xfs_repair_metadata (mnt_fd, f->scrub_type,0 , hme->e.inode.ino, hme->e.inode.gen); +/* +* Try again to find the file path, maybe we fixed the dir +* tree. +*/ +if (!hme_prefix_has_path (pfx)) { +lookup_path (ctx, hme, &new_pfx); +if (hme_prefix_has_path (&new_pfx)) + pfx = &new_pfx; + } +pthread_mutex_lock (&ctx->conlock);report_health_repair (pfx, hme, f->event_mask, outcome);pthread_mutex_unlock (&ctx->conlock);
healer/weakhandle.c +86 -0
95 lines not on this page · raw message
healer/xfs_healer.c +43 -2
64 lines not on this page · raw message
PATCH 13/28 xfs_healer: create a per-mount background monitoring service
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a systemd service definition for our self-healing filesystem daemon so that we can run it for every mounted filesystem. Add a hidden switch so that we can print the service unit name for fstests.
--- healer/xfs_healer.h | 1 healer/Makefile | 22 ++++++++ healer/system-xfs_healer.slice | 31 ++++++++++++ healer/xfs_healer.c | 16 ++++++ healer/[email protected] | 107 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 176 insertions(+), 1 deletion(-) create mode 100644 healer/system-xfs_healer.slice create mode 100644 healer/[email protected]
healer/xfs_healer.h +1 -0
@@ -26,6 +26,7 @@ struct healer_ctx {int everything;int foreground;int want_repair; +int print_svcname;/* fd and fs geometry for mount */ struct xfs_fd mnt;
healer/Makefile +21 -1
@@ -22,7 +22,23 @@ LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD) LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG) LLDFLAGS = -static -default: depend $(LTCOMMAND) +ifeq ($(HAVE_SYSTEMD),yes) [email protected] +CFLAGS += -DXFS_HEALER_SVCNAME=\"$(XFS_HEALER_SVCNAME)\" + +INSTALL_HEALER += install-systemd +SYSTEMD_SERVICES=\ + system-xfs_healer.slice \ + $(XFS_HEALER_SVCNAME) +OPTIONAL_TARGETS += $(SYSTEMD_SERVICES) +endif + +default: depend $(LTCOMMAND) $(SYSTEMD_SERVICES) + +%.service: %.service.in $(builddefs) + @echo " [SED] $@" + $(Q)$(SED) -e "s|@pkg_libexec_dir@|$(PKG_LIBEXEC_DIR)|g" \ + < $< > $@ include $(BUILDRULES)
@@ -32,6 +48,10 @@ install-healer: default $(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR) $(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR) +install-systemd: default + $(INSTALL) -m 755 -d $(SYSTEMD_SYSTEM_UNIT_DIR) + $(INSTALL) -m 644 $(SYSTEMD_SERVICES) $(SYSTEMD_SYSTEM_UNIT_DIR) + install-dev: -include .dep
healer/system-xfs_healer.slice new file +31 -0
@@ -0,0 +1,31 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# Author: Darrick J. Wong <[email protected]> + +[Unit] +Description=xfs_healer background service slice +Before=slices.target + +[Slice] + +# If the CPU usage cgroup controller is available, don't use more than 2 cores +# for all background processes. One thread to read events, another to run +# repairs. +CPUQuota=200% +CPUAccounting=true + +[Install] +# As of systemd 249, the systemd cgroupv2 configuration code will drop resource +# controllers from the root and system.slice cgroups at startup if it doesn't +# find any direct dependencies that require a given controller. Newly +# activated units with resource control directives are created under the system +# slice but do not cause a reconfiguration of the slice's resource controllers. +# Hence we cannot put CPUQuota= into the xfs_healer service units directly. +# +# For the CPUQuota directive to have any effect, we must therefore create an +# explicit definition file for the slice that systemd creates to contain the +# xfs_healer instance units (e.g. [email protected]) and we must configure +# this slice as a dependency of the system slice to establish the direct +# dependency relation. +WantedBy=system.slice
healer/xfs_healer.c +16 -0
@@ -407,6 +407,7 @@ enum long_opt_nr {LOPT_HELP ,LOPT_QUIET ,LOPT_REPAIR , +LOPT_SVCNAME , LOPT_MAX, };
@@ -438,6 +439,7 @@ main( [LOPT_HELP] = {"help", no_argument, NULL, 0 }, [LOPT_QUIET] = {"quiet", no_argument, &ctx.log, 0 }, [LOPT_REPAIR] = {"repair", no_argument, &ctx.want_repair, 1 }, + [LOPT_SVCNAME] = {"svcname", no_argument, &ctx.print_svcname, 1 }, [LOPT_MAX] = {NULL, 0, NULL, 0 }, };
@@ -474,6 +476,20 @@ main( ctx.mntpoint = argv[optind]; +if (ctx.print_svcname) { +char unitname[PATH_MAX ]; + + ret =systemd_path_instance_unit_name (XFS_HEALER_SVCNAME , + ctx.mntpoint, unitname,sizeof (unitname)); +if (ret) { +perror (ctx.mntpoint); +return EXIT_FAILURE ; + } + +printf ("%s\n" , unitname); +return EXIT_SUCCESS ; + } + ret =setup_monitor (&ctx);if (ret) goto out_events;
healer/[email protected] new file +107 -0
107 lines not on this page · raw message
Re: [PATCH 13/28] xfs_healer: create a per-mount background monitoring service
Looks good:
PATCH 14/28 xfs_healer: create a service to start the per-mount healer service
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a daemon to wait for xfs mount events via fsnotify and start up the per-mount healer service. It's important that we're running in the same mount namespace as the mount, so we're a fanotify client to avoid having to filter the mount namespaces ourselves.
--- libfrog/systemd.h | 23 +- configure.ac | 1 healer/Makefile | 16 +- healer/xfs_healer_start.c | 368 ++++++++++++++++++++++++++++++++++++ healer/xfs_healer_start.service.in | 85 ++++++++ include/builddefs.in | 5 m4/package_libcdev.m4 | 24 ++ 7 files changed, 511 insertions(+), 11 deletions(-) create mode 100644 healer/xfs_healer_start.c create mode 100644 healer/xfs_healer_start.service.in
libfrog/systemd.h +15 -8
@@ -22,6 +22,20 @@ static inline bool systemd_is_service(void)return getenv ("SERVICE_MODE" ) !=NULL ; } +/* Special processing for a service/daemon program that is exiting. */ +static inline int +systemd_service_exit_now (int ret) +{ +/* +* If we're being run as a service, the return code must fit the LSB +* init script action error guidelines, which is to say that we +* compress all errors to 1 ("generic or unspecified error", LSB 5.0 +* section 22.2) and hope the admin will scan the log for what actually +* happened. +*/ +return ret !=0 ?EXIT_FAILURE :EXIT_SUCCESS ; +} +/* Special processing for a service/daemon program that is exiting. */ static inline int systemd_service_exit (int ret)
@@ -35,14 +49,7 @@ systemd_service_exit(int ret)*/ sleep (2 ); -/* -* If we're being run as a service, the return code must fit the LSB -* init script action error guidelines, which is to say that we -* compress all errors to 1 ("generic or unspecified error", LSB 5.0 -* section 22.2) and hope the admin will scan the log for what actually -* happened. -*/ - return ret != 0 ? EXIT_FAILURE : EXIT_SUCCESS; + return systemd_service_exit_now(ret); }#endif /* __LIBFROG_SYSTEMD_H__ */
configure.ac +1 -0
@@ -194,6 +194,7 @@ if test "$have_listmount" = "yes"; then AC_HAVE_LISTMOUNT_NS_FD AC_HAVE_STATMOUNT_SUPPORTED_MASK fi +AC_HAVE_FANOTIFY_MOUNTINFO if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then AC_PACKAGE_CHECK_UBSAN
healer/Makefile +13 -3
@@ -9,6 +9,7 @@ include $(builddefs) INSTALL_HEALER = install-healer LTCOMMAND = xfs_healer +BUILD_TARGETS = $(LTCOMMAND) CFILES = \ fsrepair.c \
@@ -31,9 +32,18 @@ SYSTEMD_SERVICES=\ system-xfs_healer.slice \ $(XFS_HEALER_SVCNAME) OPTIONAL_TARGETS += $(SYSTEMD_SERVICES) -endif +endif # HAVE_SYSTEMD -default: depend $(LTCOMMAND) $(SYSTEMD_SERVICES) +ifeq ($(HAVE_HEALER_START_DEPS),yes) +BUILD_TARGETS += xfs_healer_start +SYSTEMD_SERVICES += xfs_healer_start.service +endif # xfs_healer_start deps + +default: depend $(BUILD_TARGETS) $(SYSTEMD_SERVICES) + +xfs_healer_start: $(SUBDIRS) xfs_healer_start.o $(LTDEPENDENCIES) + @echo " [LD] $@" + $(Q)$(LTLINK) -o $@ $(LDFLAGS) xfs_healer_start.o $(LDLIBS) %.service: %.service.in $(builddefs) @echo " [SED] $@"
@@ -46,7 +56,7 @@ install: $(INSTALL_HEALER) install-healer: default $(INSTALL) -m 755 -d $(PKG_LIBEXEC_DIR) - $(INSTALL) -m 755 $(LTCOMMAND) $(PKG_LIBEXEC_DIR) + $(INSTALL) -m 755 $(BUILD_TARGETS) $(PKG_LIBEXEC_DIR) install-systemd: default $(INSTALL) -m 755 -d $(SYSTEMD_SYSTEM_UNIT_DIR)
healer/xfs_healer_start.c new file +368 -0
368 lines not on this page · raw message
healer/xfs_healer_start.service.in new file +85 -0
85 lines not on this page · raw message
include/builddefs.in +5 -0
@@ -124,6 +124,7 @@ HAVE_LISTMOUNT = @have_listmount@ HAVE_LISTMOUNT_NS_FD = @have_listmount_ns_fd@ HAVE_STATMOUNT_SUPPORTED_MASK = @have_statmount_supported_mask@ NEED_INTERNAL_STATMOUNT = @need_internal_statmount@ +HAVE_FANOTIFY_MOUNTINFO = @have_fanotify_mountinfo@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall # -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
@@ -159,6 +160,10 @@ ifeq ($(HAVE_LIBURCU_ATOMIC64),yes) PCFLAGS += -DHAVE_LIBURCU_ATOMIC64 endif +ifeq ($(ENABLE_HEALER)$(HAVE_SYSTEMD)$(HAVE_LISTMOUNT)$(HAVE_FANOTIFY_MOUNTINFO),yesyesyesyes) +HAVE_HEALER_START_DEPS = yes +endif + SANITIZER_CFLAGS += @addrsan_cflags@ @threadsan_cflags@ @ubsan_cflags@ @autovar_init_cflags@ SANITIZER_LDFLAGS += @addrsan_ldflags@ @threadsan_ldflags@ @ubsan_ldflags@
m4/package_libcdev.m4 +24 -0
@@ -452,3 +452,27 @@ AC_DEFUN([AC_HAVE_STATMOUNT_SUPPORTED_MASK], AC_SUBST(have_statmount_supported_mask) AC_SUBST(need_internal_statmount) ]) + +# +# Check if fanotify will give us mount notifications (6.15). +# +AC_DEFUN([AC_HAVE_FANOTIFY_MOUNTINFO], + [AC_MSG_CHECKING([for fanotify mount events]) + AC_LINK_IFELSE( + [AC_LANG_PROGRAM([[ +#define _GNU_SOURCE +#include <stdlib.h> +#include <fcntl.h> +#include <sys/fanotify.h> + ]], [[ + struct fanotify_event_info_mnt info; + + int fan_fd = fanotify_init(FAN_REPORT_MNT, 0); + fanotify_mark(fan_fd, FAN_MARK_ADD | FAN_MARK_MNTNS, FAN_MNT_ATTACH, + -1, NULL); + ]]) + ], have_fanotify_mountinfo=yes + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no)) + AC_SUBST(have_fanotify_mountinfo) + ])
Re: [PATCH 14/28] xfs_healer: create a service to start the per-mount healer service
Looks good:
PATCH 15/28 xfs_healer: don't start service if kernel support unavailable
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Use ExecCondition= in the system service to check if kernel support for the health monitor is available. If not, we don't want to run the service, have it fail, and generate a bunch of silly log messages.
--- healer/xfs_healer.h | 1 + healer/xfs_healer.c | 47 ++++++++++++++++++++++++++++++++--------- healer/[email protected] | 1 + 3 files changed, 39 insertions(+), 10 deletions(-)
healer/xfs_healer.h +1 -0
@@ -27,6 +27,7 @@ struct healer_ctx {int foreground;int want_repair;int print_svcname; +int support_check;/* fd and fs geometry for mount */ struct xfs_fd mnt;
healer/xfs_healer.c +37 -10
104 lines not on this page · raw message
healer/[email protected] +1 -0
@@ -17,6 +17,7 @@ RequiresMountsFor=%f [Service] Type=exec Environment=SERVICE_MODE=1 +ExecCondition=@pkg_libexec_dir@/xfs_healer --supported %f ExecStart=@pkg_libexec_dir@/xfs_healer %f SyslogIdentifier=%N
PATCH 16/28 xfs_healer: use the autofsck fsproperty to select mode
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make the xfs_healer background service query the autofsck filesystem property to figure out which operating mode it should use.
--- healer/xfs_healer.h | 1 libfrog/fsproperties.h | 5 ++ healer/xfs_healer.c | 102 +++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 105 insertions(+), 3 deletions(-)
healer/xfs_healer.h +1 -0
@@ -28,6 +28,7 @@ struct healer_ctx {int want_repair;int print_svcname;int support_check; +int autofsck;/* fd and fs geometry for mount */ struct xfs_fd mnt;
libfrog/fsproperties.h +5 -0
@@ -52,6 +52,11 @@ bool fsprop_validate(const char *name, const char *value);#define FSPROP_AUTOFSCK_NAME "autofsck" +/* filesystem property name for fgetxattr */ +#define VFS_FSPROP_AUTOFSCK_NAME (FSPROP_NAMESPACE \ + FSPROP_NAME_PREFIX \ + FSPROP_AUTOFSCK_NAME) +enum fsprop_autofsck {FSPROP_AUTOFSCK_UNSET =0 ,/* do not set property */ FSPROP_AUTOFSCK_NONE ,/* no background scrubs */
healer/xfs_healer.c +99 -3
161 lines not on this page · raw message
PATCH 17/28 xfs_healer: run full scrub after lost corruption events or targeted repair failure
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If we fail to perform a spot repair of metadata or the kernel tells us that it lost corruption events due to queue limits, initiate a full run of the online fsck service to try to fix the error.
--- healer/xfs_healer.h | 3 ++ healer/Makefile | 2 + healer/fsrepair.c | 81 +++++++++++++++++++++++++++++++++++++++++++++----- healer/weakhandle.c | 13 ++++++++ healer/xfs_healer.c | 7 ++++ include/builddefs.in | 1 + scrub/Makefile | 7 ++-- 7 files changed, 102 insertions(+), 12 deletions(-)
healer/xfs_healer.h +3 -0
@@ -72,6 +72,7 @@ void lookup_path(struct healer_ctx *ctx,int repair_metadata (struct healer_ctx *ctx,const struct hme_prefix *pfx,const struct xfs_health_monitor_event *hme);bool healer_can_repair (struct healer_ctx *ctx); +void run_full_repair (struct healer_ctx *ctx);/* weakhandle.c */ int weakhandle_alloc(int fd, const char *mountpoint, const char *fsname,
@@ -80,5 +81,7 @@ int weakhandle_reopen(struct weakhandle *wh, int *fd);void weakhandle_free (struct weakhandle **whp);int weakhandle_getpath_for (struct weakhandle *wh,uint64_t ino,uint32_t gen,char *path,size_t pathlen); +int weakhandle_instance_unit_name (struct weakhandle *wh,const char *template, +char *unitname,size_t unitnamelen);#endif /* XFS_HEALER_XFS_HEALER_H_ */
healer/Makefile +2 -0
@@ -19,6 +19,8 @@ xfs_healer.c HFILES = \ xfs_healer.h +CFLAGS+=-DXFS_SCRUB_SVCNAME=\"$(XFS_SCRUB_SVCNAME)\" + LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBURCU) $(LIBPTHREAD) LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG) LLDFLAGS = -static
healer/fsrepair.c +73 -8
156 lines not on this page · raw message
healer/weakhandle.c +13 -0
@@ -13,6 +13,7 @@#include "libfrog/workqueue.h" #include "libfrog/getparents.h" #include "libfrog/paths.h" +#include "libfrog/systemd.h" #include "xfs_healer.h" struct weakhandle {
@@ -199,3 +200,15 @@ weakhandle_getpath_for(close (mnt_fd);return ret; } + +/* Compute the systemd instance unit name for this mountpoint. */ +int +weakhandle_instance_unit_name ( +struct weakhandle *wh, +const char *template, +char *unitname, +size_t unitnamelen) +{ +return systemd_path_instance_unit_name (template, wh->mntpoint, + unitname, unitnamelen); +}
healer/xfs_healer.c +7 -0
@@ -138,6 +138,13 @@ handle_event(pthread_mutex_unlock (&ctx->conlock); } +/* +* If we didn't ask for all the metadata reports (including the healthy +* ones) and the kernel tells us it lost something, run the full scan. +*/ +if (hme->type ==XFS_HEALTH_MONITOR_TYPE_LOST && !ctx->everything) +run_full_repair (ctx); +/* Initiate a repair if appropriate. */ if (will_repair)repair_metadata (ctx, &pfx, hme);
include/builddefs.in +1 -0
@@ -62,6 +62,7 @@ MKFS_CFG_DIR = @datadir@/@pkg_name@/mkfs PKG_STATE_DIR = @localstatedir@/lib/@pkg_name@ XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP=$(PKG_STATE_DIR)/xfs_scrub_all_media.stamp [email protected] CC = @cc@ BUILD_CC = @BUILD_CC@
scrub/Makefile +3 -4
@@ -8,7 +8,6 @@ include $(builddefs) SCRUB_PREREQS=$(HAVE_GETFSMAP) [email protected] [email protected] ifeq ($(SCRUB_PREREQS),yes)
@@ -21,7 +20,7 @@ XFS_SCRUB_SERVICE_ARGS = -b -o autofsck ifeq ($(HAVE_SYSTEMD),yes) INSTALL_SCRUB += install-systemd SYSTEMD_SERVICES=\ - $(scrub_svcname) \ + $(XFS_SCRUB_SVCNAME) \ [email protected] \ $(scrub_media_svcname) \ [email protected] \
@@ -123,7 +122,7 @@ xfs_scrub_all.timer: xfs_scrub_all.timer.in $(builddefs) $(XFS_SCRUB_ALL_PROG): $(XFS_SCRUB_ALL_PROG).in $(builddefs) $(TOPDIR)/libfrog/gettext.py @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \ - -e "s|@scrub_svcname@|$(scrub_svcname)|g" \ + -e "s|@scrub_svcname@|$(XFS_SCRUB_SVCNAME)|g" \ -e "s|@scrub_media_svcname@|$(scrub_media_svcname)|g" \ -e "s|@pkg_version@|$(PKG_VERSION)|g" \ -e "s|@stampfile@|$(XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP)|g" \
@@ -137,7 +136,7 @@ $(XFS_SCRUB_ALL_PROG): $(XFS_SCRUB_ALL_PROG).in $(builddefs) $(TOPDIR)/libfrog/g xfs_scrub_fail: xfs_scrub_fail.in $(builddefs) @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \ - -e "s|@scrub_svcname@|$(scrub_svcname)|g" \ + -e "s|@scrub_svcname@|$(XFS_SCRUB_SVCNAME)|g" \ -e "s|@pkg_version@|$(PKG_VERSION)|g" < $< > $@ $(Q)chmod a+x $@
PATCH 18/28 xfs_healer: use getmntent to find moved filesystems
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
It's possible that a mounted filesystem can move mountpoints between the time of the initial mount (at which point xfs_healer starts) and when it actually wants to start a repair. When this happens, weakhandle::mountpoint becomes obsolete and opening it will either fail with ENOENT or the handle revalidation will return ESTALE.
However, we do still have a means to find the mounted filesystem -- the fsname parameter (aka the path to the data device at mount time). This is record in /proc/mounts, which means that we can iterate getmntent to see if we can find the mount elsewhere.
As documented a few patches ago, this would be easier if we had revocable fds that didn't pin mounts, but that's a very huge ask.
--- healer/weakhandle.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 51 insertions(+), 4 deletions(-)
healer/weakhandle.c +51 -4
@@ -65,10 +65,14 @@ weakhandle_alloc(return -1 ; } -/* Reopen a file handle obtained via weak reference. */ -int -weakhandle_reopen ( +/* +* Reopen a file handle obtained via weak reference, using the given path to a +* mount point. +*/ +static int +weakhandle_reopen_from (struct weakhandle *wh, +const char *path,int *fd) {void *hanp;
@@ -78,7 +82,7 @@ weakhandle_reopen( *fd =-1 ; - mnt_fd =open (wh->mntpoint,O_RDONLY ); + mnt_fd =open (path,O_RDONLY );if (mnt_fd <0 )return -1 ;
@@ -102,6 +106,49 @@ weakhandle_reopen(return -1 ; } +/* Reopen a file handle obtained via weak reference. */ +int +weakhandle_reopen ( +struct weakhandle *wh, +int *fd) +{ +FILE *mtab; +struct mntent *mnt; +int ret; + +/* First try reopening using the original mountpoint */ + ret =weakhandle_reopen_from (wh, wh->mntpoint, fd); +if (!ret) +return 0 ; + +/* +* That didn't work, so now walk /proc/mounts to find a mount with the +* same fsname (aka xfs data device path) as when we started. +*/ + mtab =setmntent (_PATH_PROC_MOUNTS,"r" ); +if (!mtab) +return -1 ; + +while ((mnt =getmntent (mtab)) !=NULL ) { +if (strcmp (mnt->mnt_type,"xfs" )) +continue ; +if (strcmp (mnt->mnt_fsname, wh->fsname)) +continue ; + + ret =weakhandle_reopen_from (wh, mnt->mnt_dir, fd); +if (!ret) +break ; + } + +if (*fd <0 ) { + errno =ESTALE ; + ret =-1 ; + } + +endmntent (mtab); +return ret; +} +/* Tear down a weak handle */ void weakhandle_free(
PATCH 19/28 xfs_healer: use statmount to find moved filesystems even faster
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
As noted in the previous patch, it's possible that a mounted filesystem can move mountpoints between the time of the initial mount (at which point xfs_healer starts) and when it actually wants to start a repair. The previous patch fixed that problem by using getmntent to walk /proc/self/mounts to see if it finds a mount with the same "source" name, aka data device.
However, this is really slow if there are a lot of filesystems because we end up wading through a lot of irrelevant information. However, statmount() can help us here because as of Linux 7.0 we can open the passed-in path at startup, call statmount() on it to retrieve the mnt_id, and then call it again later with that same mnt_id to find the mountpoint. Luckily xfs_healthmon didn't get merged until 7.0 so it's more or less guaranteed to be there if XFS_IOC_HEALTH_MONITOR succeeds.
Obviously if this doesn't work, we can fall back to the slow walk.
--- healer/xfs_healer.h | 7 +++++-- healer/weakhandle.c | 24 ++++++++++++++++++++++++ healer/xfs_healer.c | 37 +++++++++++++++++++++++++++++++++++-- 3 files changed, 64 insertions(+), 4 deletions(-)
healer/xfs_healer.h +5 -2
@@ -39,6 +39,9 @@ struct healer_ctx {/* Shared reference to the getmntent fsname for reconnecting */ const char *fsname; +/* Mount id for faster reconnecting */ +uint64_t mnt_id; +/* weak file handle so we can reattach to filesystem */ struct weakhandle *wh;
@@ -75,8 +78,8 @@ bool healer_can_repair(struct healer_ctx *ctx);void run_full_repair (struct healer_ctx *ctx);/* weakhandle.c */ -int weakhandle_alloc (int fd,const char *mountpoint,const char *fsname, -struct weakhandle **whp); +int weakhandle_alloc (int fd,const char *mountpoint,uint64_t mnt_id, +const char *fsname,struct weakhandle **whp);int weakhandle_reopen (struct weakhandle *wh,int *fd);void weakhandle_free (struct weakhandle **whp); int weakhandle_getpath_for(struct weakhandle *wh, uint64_t ino, uint32_t gen,
healer/weakhandle.c +24 -0
@@ -14,6 +14,7 @@#include "libfrog/getparents.h" #include "libfrog/paths.h" #include "libfrog/systemd.h" +#include "libfrog/statmount.h" #include "xfs_healer.h" struct weakhandle {
@@ -23,6 +24,9 @@ struct weakhandle {/* Shared reference to the getmntent fsname for reconnecting */ const char *fsname; +/* Mount id for faster reconnecting */ +uint64_t mnt_id; +/* handle to root dir */ void *hanp;size_t hlen;
@@ -33,6 +37,7 @@ intweakhandle_alloc ( int fd, const char *mountpoint, + uint64_t mnt_id, const char *fsname, struct weakhandle **whp) {
@@ -51,6 +56,7 @@ weakhandle_alloc(return -1 ; wh->mntpoint = mountpoint; + wh->mnt_id = mnt_id; wh->fsname = fsname; ret =fd_to_handle (fd, &wh->hanp, &wh->hlen);
@@ -112,6 +118,9 @@ weakhandle_reopen(struct weakhandle *wh, int *fd) { + const size_t smbuf_size = +libfrog_statmount_sizeof (PATH_MAX ); +struct statmount *smbuf =alloca (smbuf_size);FILE *mtab;struct mntent *mnt;int ret;
@@ -121,6 +130,21 @@ weakhandle_reopen(if (!ret)return 0 ; +/* +* The original mountpoint didn't work, which means the mount might +* have been moved. Look up the mountpoint for the mount id that we +* captured earlier, which is a quick lookup if there are many mounts. +* Note that @ret is nonzero here. +*/ + ret =libfrog_statmount (wh->mnt_id,DEFAULT_MOUNTNS_FD , +STATMOUNT_MNT_POINT , smbuf, smbuf_size); +if (ret || !(smbuf->mask &STATMOUNT_MNT_POINT )) + goto fallback; + ret =weakhandle_reopen_from (wh, smbuf->str + smbuf->mnt_point, fd); +if (!ret) +return 0 ; + +fallback:/* * That didn't work, so now walk /proc/mounts to find a mount with the * same fsname (aka xfs data device path) as when we started.
healer/xfs_healer.c +35 -2
@@ -15,6 +15,7 @@#include "libfrog/workqueue.h" #include "libfrog/systemd.h" #include "libfrog/fsproperties.h" +#include "libfrog/statmount.h" #include "xfs_healer.h" /* Program name; needed for libfrog error reports. */
@@ -163,11 +164,43 @@ try_capture_fsinfo( {struct mntent *mnt;FILE *mtp; -char rpath[PATH_MAX ], rmnt_dir[PATH_MAX ]; +const size_t smbuf_size = +libfrog_statmount_sizeof (PATH_MAX +128 ); +struct statmount *smbuf =alloca (smbuf_size); +char *rmnt_dir = smbuf->str; +char rpath[PATH_MAX ]; +int ret;if (!realpath (ctx->mntpoint, rpath))return -1 ; +/* +* In Linux 7.0 we can do statmount on an open file, which means that +* we can capture the mnt_id, mount point, and fsname, which can help +* us find a mount --move'd elsewhere in the directory tree. +*/ + ret =libfrog_fstatmount (ctx->mnt.fd,STATMOUNT_MNT_POINT , smbuf, + smbuf_size); +if (ret || !(smbuf->mask &STATMOUNT_MNT_POINT )) + goto fallback; +if (strcmp (rpath, smbuf->str + smbuf->mnt_point)) + goto fallback; + + ret =libfrog_fstatmount (ctx->mnt.fd, +STATMOUNT_SB_SOURCE |STATMOUNT_MNT_BASIC , + smbuf, smbuf_size); +if (ret || !(smbuf->mask &STATMOUNT_SB_SOURCE )) + goto fallback; + + ctx->fsname =strdup (smbuf->str + smbuf->sb_source); + ctx->mnt_id = smbuf->mnt_id; +return 0 ; + +fallback: +/* +* If statmount isn't available for whatever reason, fall back to +* walking the mount table via getmntent. +*/ mtp =setmntent (_PATH_PROC_MOUNTS,"r" );if (mtp ==NULL )return -1 ;
@@ -341,7 +374,7 @@ setup_monitor(* paths for logging. */ if (ctx->want_repair ||healer_has_parent (ctx)) { - ret =weakhandle_alloc (ctx->mnt.fd, ctx->mntpoint, + ret =weakhandle_alloc (ctx->mnt.fd, ctx->mntpoint, ctx->mnt_id, ctx->fsname, &ctx->wh);if (ret) { fprintf(stderr, "%s: %s: %s\n", ctx->mntpoint,
Re: [PATCH 19/28] xfs_healer: use statmount to find moved filesystems even faster
Quoted 10 lines
However, this is really slow if there are a lot of filesystems because we end up wading through a lot of irrelevant information. However,
statmount()can help us here because as of Linux 7.0 we can open the passed-in path at startup, callstatmount()on it to retrieve themnt_id, and then call it again later with that samemnt_idto find the mountpoint. Luckilyxfs_healthmondidn't get merged until 7.0 so it's more or less guaranteed to be there ifXFS_IOC_HEALTH_MONITORsucceeds.Obviously if this doesn't work, we can fall back to the slow walk.
Can we kill the fallback and instead just have a good error message if someone messes up their backports?
Re: [PATCH 19/28] xfs_healer: use statmount to find moved filesystems even faster
Quoted 9 lines · Christoph Hellwig
Quoted 9 lines
However, this is really slow if there are a lot of filesystems because we end up wading through a lot of irrelevant information. However,
statmount()can help us here because as of Linux 7.0 we can open the passed-in path at startup, callstatmount()on it to retrieve themnt_id, and then call it again later with that samemnt_idto find the mountpoint. Luckilyxfs_healthmondidn't get merged until 7.0 so it's more or less guaranteed to be there ifXFS_IOC_HEALTH_MONITORsucceeds.
Quoted 4 lines
Quoted 1 line
Obviously if this doesn't work, we can fall back to the slow walk.
Can we kill the fallback and instead just have a good error message if someone messes up their backports?
Yes we could, though the risk with dropping the previous patch is that someone could introduce a third-party seccomp policy that forbids statmount() even on kernels that support it, and then xfs_healer would just fail if the mount moves.
That would be totally stupid because the only reason you'd encounter that is because either (a) your distro screwed up their security policy or (b) your paranoid IT department deploys an Enterprise Security Model written by a ven-duh who hasn't gotten around to evaluating the new syscalls and blocks anything they've not read about.
I'd rather keep the getmntent stuff around since that's the classic way linux programs (including xfsprogs) have handled scanning the mount tables. But if you feel strongly about not having getmntent it's not hard to pull it out.
On a practical note: We have 5 weeks more to wait to be sure that STATMOUNT_BY_FD actually makes it into a released kernel. Seeing as xfsprogs 7.0 won't get tagged until well after that, we don't have to decide to strip out getmntent anytime soon.
--D
Re: [PATCH 19/28] xfs_healer: use statmount to find moved filesystems even faster
Quoted 26 lines · Darrick J. Wong
Quoted 9 lines · Christoph Hellwig
Quoted 9 lines
However, this is really slow if there are a lot of filesystems because we end up wading through a lot of irrelevant information. However,
statmount()can help us here because as of Linux 7.0 we can open the passed-in path at startup, callstatmount()on it to retrieve themnt_id, and then call it again later with that samemnt_idto find the mountpoint. Luckilyxfs_healthmondidn't get merged until 7.0 so it's more or less guaranteed to be there ifXFS_IOC_HEALTH_MONITORsucceeds.Quoted 4 lines
Quoted 1 line
Obviously if this doesn't work, we can fall back to the slow walk.
Can we kill the fallback and instead just have a good error message if someone messes up their backports?
Yes we could, though the risk with dropping the previous patch is that someone could introduce a third-party seccomp policy that forbids
statmount()even on kernels that support it, and thenxfs_healerwould just fail if the mount moves.That would be totally stupid because the only reason you'd encounter that is because either (a) your distro screwed up their security policy or (b) your paranoid IT department deploys an Enterprise Security Model written by a ven-duh who hasn't gotten around to evaluating the new syscalls and blocks anything they've not read about.
Quoted 4 lines
I'd rather keep the getmntent stuff around since that's the classic way linux programs (including xfsprogs) have handled scanning the mount tables. But if you feel strongly about not having getmntent it's not hard to pull it out.
Actually, ignore all this, I found a much better reason for keeping both versions:
Each bind mount gets a new mnt_id, so getmntent is the only way to find a different vfsmount of the same filesystem. If you do this:
mount /dev/sda /mntxfs_healer /mnt &
mount /mnt /opt --bind
umount /mnt
the filesystem itself never gets unmounted, but statmount() won't be able to find /opt without scanning the mount table. Observe that the bind mount gets a different vfsmount object, and hence a different mnt_id:
# mount /dev/sda /mnt
# xfs_io -c statmount /mnt | grep mnt_id:
mnt_id: 0x80004775
# mount /mnt /opt --bind
# xfs_io -c statmount /opt | grep mnt_id:
mnt_id: 0x8000477aThe statmount reconnection thing, however, works for mount --move because the vfsmount object doesn't go away, it merely moves around:
# mount -t tmpfs urk /mnt
# mount --make-rprivate /mnt
# mkdir -p /mnt/a /mnt/b
# mount /dev/sda /mnt/a
# xfs_io -c statmount /mnt/a | grep mnt_id:
mnt_id: 0x800048a9
# mount --move /mnt/a /mnt/b
# xfs_io -c statmount /mnt/b | grep mnt_id:
mnt_id: 0x800048a9I'm not sure when anyone would really do this, but it's certainly possible to do this, and every other process in the same mount namespace will see it. I'll go write a new test to exercise this, since I forgot to do that earlier.
--D
Re: [PATCH 19/28] xfs_healer: use statmount to find moved filesystems even faster
In reply to Darrick J. WongQuoted 4 lines · Darrick J. Wong
Yes we could, though the risk with dropping the previous patch is that someone could introduce a third-party seccomp policy that forbids
statmount()even on kernels that support it, and thenxfs_healerwould just fail if the mount moves.
Trying toi protect against broken third party security policies is impossible. If people do cargo cult "security" they get what they pay for.
Quoted 4 lines
I'd rather keep the getmntent stuff around since that's the classic way linux programs (including xfsprogs) have handled scanning the mount tables. But if you feel strongly about not having getmntent it's not hard to pull it out.
I'm not a fan of keeping code around that doesn't really get exercised. Although your next mail suggest we could exercise it, so maybe we just need goot test for both cases.
I still hate the fragility of all this, but I can't immediately thing of a better way.
Re: [PATCH 19/28] xfs_healer: use statmount to find moved filesystems even faster
Quoted 15 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
Yes we could, though the risk with dropping the previous patch is that someone could introduce a third-party seccomp policy that forbids
statmount()even on kernels that support it, and thenxfs_healerwould just fail if the mount moves.Trying toi protect against broken third party security policies is impossible. If people do cargo cult "security" they get what they pay for.
Quoted 4 lines
I'd rather keep the getmntent stuff around since that's the classic way linux programs (including xfsprogs) have handled scanning the mount tables. But if you feel strongly about not having getmntent it's not hard to pull it out.
Quoted 6 lines
I'm not a fan of keeping code around that doesn't really get exercised. Although your next mail suggest we could exercise it, so maybe we just need goot test for both cases.
I still hate the fragility of all this, but I can't immediately thing of a better way.
I'll think about this more today, but my gut feeling is that the getmntent patch should stay because it works for both cases. But let me post my new testcase exercising mount --move first.
--D
PATCH 20/28 xfs_healer: validate that repair fds point to the monitored fs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When xfs_healer reopens a mountpoint to perform a repair, it should validate that the opened fd points to a file on the same filesystem as the one being monitored.
--- healer/xfs_healer.h | 4 +++- healer/fsrepair.c | 18 +++++++++++++++++- healer/weakhandle.c | 23 +++++++++++++++++------ 3 files changed, 37 insertions(+), 8 deletions(-)
healer/xfs_healer.h +3 -1
@@ -80,7 +80,9 @@ void run_full_repair(struct healer_ctx *ctx);/* weakhandle.c */ int weakhandle_alloc (int fd,const char *mountpoint,uint64_t mnt_id,const char *fsname,struct weakhandle **whp); -int weakhandle_reopen (struct weakhandle *wh,int *fd); +typedef bool (*weakhandle_fd_t )(int mnt_fd,void *data); +int weakhandle_reopen (struct weakhandle *wh,int *fd, +weakhandle_fd_t is_acceptable,void *data);void weakhandle_free (struct weakhandle **whp);int weakhandle_getpath_for (struct weakhandle *wh,uint64_t ino,uint32_t gen,char *path,size_t pathlen);
healer/fsrepair.c +17 -1
@@ -233,6 +233,22 @@ try_repair_inode(return REPAIR_DONE ; } +/* Make sure the reopened file is on the same fs as the monitor. */ +static bool +is_same_fs ( +int mnt_fd, +void *data) +{ +struct xfs_health_file_on_monitored_fs hms = { + .fd = mnt_fd, + }; +FILE *mon_fp = data; +int ret; + + ret =ioctl (fileno (mon_fp),XFS_IOC_HEALTH_FD_ON_MONITORED_FS , &hms); +return ret ==0 ; +} +/* Repair a metadata corruption. */ int repair_metadata(
@@ -244,7 +260,7 @@ repair_metadata(int repair_fd;int ret; - ret =weakhandle_reopen (ctx->wh, &repair_fd); + ret =weakhandle_reopen (ctx->wh, &repair_fd, is_same_fs, ctx->mon_fp);if (ret) { fprintf(stderr, "%s: %s: %s\n", ctx->mntpoint, _("cannot open filesystem to repair"),
healer/weakhandle.c +17 -6
@@ -79,7 +79,9 @@ static intweakhandle_reopen_from ( struct weakhandle *wh, const char *path, - int *fd) + int *fd, + weakhandle_fd_t is_acceptable, + void *data) { void *hanp;size_t hlen;
@@ -101,6 +103,11 @@ weakhandle_reopen_from( goto out_handle; } +if (is_acceptable && !is_acceptable (mnt_fd, data)) { + errno =ESTALE ; + goto out_handle; + } +free_handle (hanp, hlen); *fd = mnt_fd;return 0 ;
@@ -116,7 +123,9 @@ weakhandle_reopen_from(int weakhandle_reopen (struct weakhandle *wh, -int *fd) +int *fd, +weakhandle_fd_t is_acceptable, +void *data) {const size_t smbuf_size =libfrog_statmount_sizeof (PATH_MAX );
@@ -126,7 +135,7 @@ weakhandle_reopen(int ret;/* First try reopening using the original mountpoint */ - ret =weakhandle_reopen_from (wh, wh->mntpoint, fd); + ret =weakhandle_reopen_from (wh, wh->mntpoint, fd, is_acceptable, data);if (!ret)return 0 ;
@@ -140,7 +149,8 @@ weakhandle_reopen( STATMOUNT_MNT_POINT, smbuf, smbuf_size);if (ret || !(smbuf->mask &STATMOUNT_MNT_POINT )) goto fallback; - ret =weakhandle_reopen_from (wh, smbuf->str + smbuf->mnt_point, fd); + ret =weakhandle_reopen_from (wh, smbuf->str + smbuf->mnt_point, fd, + is_acceptable, data);if (!ret)return 0 ;
@@ -159,7 +169,8 @@ weakhandle_reopen(if (strcmp (mnt->mnt_fsname, wh->fsname))continue ; - ret =weakhandle_reopen_from (wh, mnt->mnt_dir, fd); + ret =weakhandle_reopen_from (wh, mnt->mnt_dir, fd, + is_acceptable, data);if (!ret)break ; }
@@ -244,7 +255,7 @@ weakhandle_getpath_for( fakehandle.ha_fid.fid_ino = ino; fakehandle.ha_fid.fid_gen = gen; - ret =weakhandle_reopen (wh, &mnt_fd); + ret =weakhandle_reopen (wh, &mnt_fd,NULL ,NULL );if (ret)return ret;
Re: [PATCH 20/28] xfs_healer: validate that repair fds point to the monitored fs
Looks good:
PATCH 21/28 xfs_healer: add a manual page
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a new section 8 manpage for this service daemon so others can read about what this program is supposed to do.
--- man/man8/Makefile | 40 +++++++++++++--- man/man8/xfs_healer.8 | 109 +++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_healer_start.8 | 37 +++++++++++++++ 3 files changed, 180 insertions(+), 6 deletions(-) create mode 100644 man/man8/xfs_healer.8 create mode 100644 man/man8/xfs_healer_start.8
man/man8/Makefile +34 -6
@@ -7,13 +7,41 @@ include $(TOPDIR)/include/builddefs MAN_SECTION = 8 -ifneq ("$(ENABLE_SCRUB)","yes") - MAN_PAGES = $(filter-out xfs_scrub%,$(shell echo *.$(MAN_SECTION))) -else - MAN_PAGES = $(shell echo *.$(MAN_SECTION)) - MAN_PAGES += xfs_scrub_all.8 +MAN_PAGES = \ + fsck.xfs.8 \ + mkfs.xfs.8 \ + xfs_admin.8 \ + xfs_bmap.8 \ + xfs_copy.8 \ + xfs_db.8 \ + xfs_estimate.8 \ + xfs_freeze.8 \ + xfs_fsr.8 \ + xfs_growfs.8 \ + xfs_info.8 \ + xfs_io.8 \ + xfs_logprint.8 \ + xfs_mdrestore.8 \ + xfs_metadump.8 \ + xfs_mkfile.8 \ + xfs_ncheck.8 \ + xfs_property.8 \ + xfs_protofile.8 \ + xfs_quota.8 \ + xfs_repair.8 \ + xfs_rtcp.8 \ + xfs_spaceman.8 + +ifeq ($(ENABLE_HEALER),yes) + MAN_PAGES += xfs_healer.8 endif -MAN_PAGES += mkfs.xfs.8 +ifeq ($(HAVE_HEALER_START_DEPS),yes) + MAN_PAGES += xfs_healer_start.8 +endif +ifeq ($(ENABLE_SCRUB),yes) + MAN_PAGES += xfs_scrub.8 xfs_scrub_all.8 +endif + MAN_DEST = $(PKG_MAN_DIR)/man$(MAN_SECTION) LSRCFILES = $(MAN_PAGES) DIRT = mkfs.xfs.8 xfs_scrub_all.8
man/man8/xfs_healer.8 new file +109 -0
109 lines not on this page · raw message
man/man8/xfs_healer_start.8 new file +37 -0
@@ -0,0 +1,37 @@ +.TH xfs_healer_start 8 +.SH NAME +xfs_healer_start \- starts xfs_healer instances +.SH SYNOPSIS +.B xfs_healer_start +[ +.B OPTIONS +] +.br +.B xfs_healer \-V +.SH DESCRIPTION +.B xfs_healer_start +starts the xfs_healer service whenever the kernel mounts an XFS filesystem in +the current mount namespace. +.PP +.B WARNING! +This program is +.BR EXPERIMENTAL "," +which means that its behavior and interface +could change at any time! + +Normally this program runs as a systemd service. + +.SH OPTIONS +.TP +.B \-\-supported +Check if the kernel supports listening for mount events. +Exits with 0 if it does, and non-zero if not. +.TP +.BI "\-\-mountns " path +Monitor the given mount namespace. +Defaults to the mount namespace associated with the process itself. +.TP +.BI \-V +Prints the version number and exit. +.SH SEE ALSO +.BR xfs_healer (8).
PATCH 22/28 xfs_scrub: use the verify media ioctl during phase 6 if possible
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the kernel suppots the XFS_IOC_VERIFY_MEDIA ioctl, use that to perform the phase 6 media scan instead of pwrite or the SCSI VERIFY command. This enables better integration with xfs_healer and fsnotify; and reduces the amount of work that userspace has to do.
--- scrub/disk.h | 11 ++++++++++- scrub/disk.c | 38 +++++++++++++++++++++++++++++++++++++- scrub/phase1.c | 25 +++++++++++++++++++++++++ scrub/read_verify.c | 2 +- 4 files changed, 73 insertions(+), 3 deletions(-)
scrub/disk.h +10 -1
@@ -10,18 +10,27 @@struct disk {struct stat d_sb;int d_fd; +int d_verify_fd;unsigned int d_lbalog;unsigned int d_lbasize;/* bytes */ unsigned int d_flags;unsigned int d_blksize;/* bytes */ uint64_t d_size;/* bytes */ uint64_t d_start;/* bytes */ +unsigned int d_verify_disk; };unsigned int disk_heads (struct disk *disk);struct disk *disk_open (const char *pathname);int disk_close (struct disk *disk);ssize_t disk_read_verify (struct disk *disk,void *buf,uint64_t startblock, -uint64_t blockcount); +uint64_t blockcount,bool single_step); + +static inline void +disk_config_xfs_verify (struct disk *disk,int mnt_fd,unsigned int verify_disk) +{ + disk->d_verify_fd = mnt_fd; + disk->d_verify_disk = verify_disk; +}#endif /* XFS_SCRUB_DISK_H_ */
scrub/disk.c +37 -1
@@ -190,6 +190,7 @@ disk_open( disk =calloc (1 ,sizeof (struct disk ));if (!disk)return NULL ; + disk->d_verify_fd =-1 ; disk->d_fd =open (pathname,O_RDONLY |O_DIRECT |O_NOATIME );if (disk->d_fd <0 )
@@ -323,13 +324,45 @@ disk_simulate_read_error(return 0 ; } +/* Use the XFS media verification ioctl to do the media scan */ +static ssize_t +disk_ioctl_verify ( +struct disk *disk, +uint64_t start, +uint64_t length, +bool single_step) +{ +const uint64_t orig_start_daddr =BTOBBT (start); +struct xfs_verify_media me = { + .me_start_daddr = orig_start_daddr, + .me_end_daddr =BTOBB (start + length), + .me_dev = disk->d_verify_disk, + .me_rest_us = bg_mode >2 ? bg_mode -1 :0 , + }; +int ret; + +if (single_step) + me.me_flags |=XFS_VERIFY_MEDIA_REPORT ; + + ret =ioctl (disk->d_verify_fd,XFS_IOC_VERIFY_MEDIA , &me); +if (ret <0 ) +return ret; +if (me.me_ioerror) { + errno = me.me_ioerror; +return -1 ; + } + +return BBTOB (me.me_start_daddr - orig_start_daddr); +} +/* Read-verify an extent of a disk device. */ ssize_t disk_read_verify (struct disk *disk,void *buf,uint64_t start, -uint64_t length) +uint64_t length, +bool single_step) {if (debug) {int ret;
@@ -345,6 +378,9 @@ disk_read_verify(return length; } +if (disk->d_verify_fd >=0 ) +return disk_ioctl_verify (disk, start, length, single_step); +/* Convert to logical block size. */ if (disk->d_flags &DISK_FLAG_SCSI_VERIFY ) return disk_scsi_verify(disk, BTOLBAT(disk, start),
scrub/phase1.c +25 -0
@@ -213,6 +213,29 @@ mode_from_autofsck( goto summarize; } +/* Does the XFS driver support media scanning its own disks? */ +static void +configure_xfs_verify ( +struct scrub_ctx *ctx) +{ +struct xfs_verify_media me = { + .me_start_daddr =1 , + .me_end_daddr =0 , + .me_dev =XFS_DEV_DATA , + }; +int ret; + + ret =ioctl (ctx->mnt.fd,XFS_IOC_VERIFY_MEDIA , &me); +if (ret <0 ) +return ; + +disk_config_xfs_verify (ctx->datadev, ctx->mnt.fd,XFS_DEV_DATA ); +if (ctx->logdev) +disk_config_xfs_verify (ctx->logdev, ctx->mnt.fd,XFS_DEV_LOG ); +if (ctx->rtdev) +disk_config_xfs_verify (ctx->rtdev, ctx->mnt.fd,XFS_DEV_RT ); +} +/* * Bind to the mountpoint, read the XFS geometry, bind to the block devices. * Anything we've already built will be cleaned up by scrub_cleanup.
@@ -379,6 +402,8 @@ _("Unable to find realtime device path.")); } } +configure_xfs_verify (ctx); +/* * Everything's set up, which means any failures recorded after * this point are most probably corruption errors (as opposed to
scrub/read_verify.c +1 -1
@@ -201,7 +201,7 @@ read_verify(dbg_printf ("diskverify %d %" PRIu64" %zu\n" , rvp->disk->d_fd, rv->io_start, len); sz =disk_read_verify (rvp->disk, rvp->readbuf, rv->io_start, - len); + len, io_max_size <= rvp->miniosz);if (sz == len && io_max_size < rvp->miniosz) {/* * If the verify request was 100% successful and less
Re: [PATCH 22/28] xfs_scrub: use the verify media ioctl during phase 6 if possible
Looks good:
PATCH 23/28 xfs_scrub: perform media scanning of the log region
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Scan the log area for media errors because a defect in a region could prevent the user from being able to perform log recovery.
--- scrub/phase6.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
scrub/phase6.c +8 -3
@@ -616,9 +616,14 @@ check_rmap( map->fmr_flags);/* "Unknown" extents should be verified; they could be data. */ -if ((map->fmr_flags &FMR_OF_SPECIAL_OWNER ) && - map->fmr_owner ==XFS_FMR_OWN_UNKNOWN ) - map->fmr_flags &= ~FMR_OF_SPECIAL_OWNER ; +if ((map->fmr_flags &FMR_OF_SPECIAL_OWNER )) { +switch (map->fmr_owner) { +case XFS_FMR_OWN_UNKNOWN : +case XFS_FMR_OWN_LOG : + map->fmr_flags &= ~FMR_OF_SPECIAL_OWNER ; +break ; + } + }/* * We only care about read-verifying data extents that have been
PATCH 24/28 xfs_scrub: print systemd service names
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a hidden switch to xfs_scrub to emit systemd service names for XFS services targetting filesystems paths instead of opencoding the computation in things like fstests.
--- scrub/xfs_scrub.h | 3 +++ scrub/Makefile | 7 +++++-- scrub/xfs_scrub.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 54 insertions(+), 6 deletions(-)
scrub/xfs_scrub.h +3 -0
@@ -108,6 +108,9 @@ struct scrub_ctx {* this much space per volume. */ double fstrim_block_pct; + +/* CLI options, must be int */ +int print_svcname; };/*
scrub/Makefile +5 -2
@@ -8,9 +8,12 @@ include $(builddefs) SCRUB_PREREQS=$(HAVE_GETFSMAP) [email protected] [email protected] ifeq ($(SCRUB_PREREQS),yes) +CFLAGS+=-DXFS_SCRUB_SVCNAME=\"$(XFS_SCRUB_SVCNAME)\" +CFLAGS+=-DXFS_SCRUB_MEDIA_SVCNAME=\"$(XFS_SCRUB_MEDIA_SVCNAME)\" + LTCOMMAND = xfs_scrub INSTALL_SCRUB = install-scrub XFS_SCRUB_ALL_PROG = xfs_scrub_all.py
@@ -22,7 +25,7 @@ INSTALL_SCRUB += install-systemd SYSTEMD_SERVICES=\ $(XFS_SCRUB_SVCNAME) \ [email protected] \ - $(scrub_media_svcname) \ + $(XFS_SCRUB_MEDIA_SVCNAME) \ [email protected] \ xfs_scrub_all.service \ xfs_scrub_all_fail.service \
scrub/xfs_scrub.c +46 -4
80 lines not on this page · raw message
Re: [PATCH 24/28] xfs_scrub: print systemd service names
Looks good:
PATCH 25/28 xfs_io: add listmount and statmount commands
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add two new commands: one to list all mounts via statmount, now that we use this in xfs_healer_start, and another to statmount each open file.
--- io/io.h | 6 + io/Makefile | 5 + io/init.c | 1 io/listmount.c | 361 +++++++++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_io.8 | 66 ++++++++++ 5 files changed, 439 insertions(+) create mode 100644 io/listmount.c
io/io.h +6 -0
@@ -164,3 +164,9 @@ void fsprops_init(void);void aginfo_init (void );void healthmon_init (void );void verifymedia_init (void ); + +#ifdef HAVE_LISTMOUNT +void listmount_init (void ); +#else +# define listmount_init () do { } while (0) +#endif
io/Makefile +5 -0
@@ -90,6 +90,11 @@ ifeq ($(HAVE_GETFSMAP),yes) CFILES += fsmap.c endif +ifeq ($(HAVE_LISTMOUNT),yes) +CFILES += listmount.c +LCFLAGS += -DHAVE_LISTMOUNT +endif + default: depend $(LTCOMMAND) include $(BUILDRULES)
io/init.c +1 -0
@@ -94,6 +94,7 @@ init_commands(void)fsprops_init ();healthmon_init ();verifymedia_init (); +listmount_init (); }/*
io/listmount.c new file +361 -0
361 lines not on this page · raw message
man/man8/xfs_io.8 +66 -0
72 lines not on this page · raw message
Re: [PATCH 25/28] xfs_io: add listmount and statmount commands
Looks good:
PATCH 26/28 mkfs: enable online repair if all backrefs are enabled
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If all backreferences are enabled in the filesystem, then enable online repair by default if the user didn't supply any other autofsck setting. Users might as well get full self-repair capability if they're paying for the extra metadata.
Note that it's up to each distro to enable the systemd services according to their own service activation policies. Debian policy is to enable all systemd services at package installation but they don't enable online fsck in their Kconfig so the services won't activate. RHEL and SUSE policy requires sysadmins to enable them explicitly unless the OS vendor also ships a systemd preset file enabling the services. Distros without systemd won't get any of the systemd services, obviously.
--- mkfs/xfs_mkfs.c | 9 +++++++++ 1 file changed, 9 insertions(+)
mkfs/xfs_mkfs.c +9 -0
@@ -6296,6 +6296,15 @@ main(if (mp->m_sb.sb_agcount >1 )rewrite_secondary_superblocks (mp); +/* +* If the filesystem has full backreferences and the user didn't +* express an autofsck preference, enable online repair because they +* might as well get some useful functionality from the extra metadata. +*/ +if (cli.autofsck ==FSPROP_AUTOFSCK_UNSET && + cli.sb_feat.rmapbt && cli.sb_feat.parent_pointers) + cli.autofsck =FSPROP_AUTOFSCK_REPAIR ; +if (cli.autofsck !=FSPROP_AUTOFSCK_UNSET )set_autofsck (mp, &cli);
Re: [PATCH 26/28] mkfs: enable online repair if all backrefs are enabled
Looks good:
Re: [PATCH 26/28] mkfs: enable online repair if all backrefs are enabled
In reply to Darrick J. WongIt looks like this makes xfs/805 unhappy, as it does not expect autofsck to be set by default.
Quoted 15 lines
1 file changed, 9 insertions(+)mkfs/xfs_mkfs.c +2
-0@@ -6296,6 +6296,15 @@ main(if (mp->m_sb.sb_agcount >1 )rewrite_secondary_superblocks (mp); +/* +* If the filesystem has full backreferences and the user didn't
mkfs/xfs_mkfs.c main( · line 6310
Quoted 10 lines
mkfs/xfs_mkfs.c +7
-0+* express an autofsck preference, enable online repair because they +* might as well get some useful functionality from the extra metadata. +*/ +if (cli.autofsck ==FSPROP_AUTOFSCK_UNSET && + cli.sb_feat.rmapbt && cli.sb_feat.parent_pointers) + cli.autofsck =FSPROP_AUTOFSCK_REPAIR ; +if (cli.autofsck !=FSPROP_AUTOFSCK_UNSET )set_autofsck (mp, &cli);
---end quoted text---
Re: [PATCH 26/28] mkfs: enable online repair if all backrefs are enabled
Quoted 2 lines · Christoph Hellwig
It looks like this makes xfs/805 unhappy, as it does not expect autofsck to be set by default.
Yeah. I've a patch to remove the "no autofsck option at all" test because you can't really change the default without breaking old xfsprogs, and a bunch of sed to fake the output seems pretty silly.
--D
Quoted 27 lines
Quoted 15 lines
1 file changed, 9 insertions(+)mkfs/xfs_mkfs.c +2
-0@@ -6296,6 +6296,15 @@ main(if (mp->m_sb.sb_agcount >1 )rewrite_secondary_superblocks (mp); +/* +* If the filesystem has full backreferences and the user didn't mkfs/xfs_mkfs.c main( · line 6310
Quoted 10 lines
mkfs/xfs_mkfs.c +7
-0+* express an autofsck preference, enable online repair because they +* might as well get some useful functionality from the extra metadata. +*/ +if (cli.autofsck ==FSPROP_AUTOFSCK_UNSET && + cli.sb_feat.rmapbt && cli.sb_feat.parent_pointers) + cli.autofsck =FSPROP_AUTOFSCK_REPAIR ; +if (cli.autofsck !=FSPROP_AUTOFSCK_UNSET )set_autofsck (mp, &cli);---end quoted text---
PATCH 27/28 debian/control: listify the build dependencies
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
This will make it less gross to add more build deps later.
--- debian/control | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
debian/control +13 -1
@@ -3,7 +3,19 @@ Section: admin Priority: optional Maintainer: XFS Development Team <[email protected]> Uploaders: Nathan Scott <[email protected]>, Anibal Monsalve Salazar <[email protected]> -Build-Depends: libinih-dev (>= 53), uuid-dev, debhelper (>= 12), gettext, libtool, libedit-dev, libblkid-dev (>= 2.17), linux-libc-dev, libdevmapper-dev, libicu-dev, pkg-config, liburcu-dev, systemd-dev | systemd (<< 253-2~) +Build-Depends: debhelper (>= 12), + gettext, + libblkid-dev (>= 2.17), + libdevmapper-dev, + libedit-dev, + libicu-dev, + libinih-dev (>= 53), + libtool, + liburcu-dev, + linux-libc-dev, + pkg-config, + systemd-dev | systemd (<< 253-2~), + uuid-dev Standards-Version: 4.0.0 Homepage: https://xfs.wiki.kernel.org/
Re: [PATCH 27/28] debian/control: listify the build dependencies
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This will make it less gross to add more build deps later.
Looks good:
PATCH 28/28 debian: enable xfs_healer on the root filesystem by default
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now that we're finished building autonomous repair, enable the healer service on the root filesystem by default. The root filesystem is mounted by the initrd prior to starting systemd, which is why the xfs_healer_start service cannot autostart the service for the root filesystem.
dh_installsystemd won't activate a template service (aka one with an at-sign in the name) even if it provides a DefaultInstance directive to make that possible. Hence we enable this explicitly via the postinst script.
Note that Debian enables services by default upon package installation, so this is consistent with their policies. Their kernel doesn't enable online fsck, so healer won't do much more than monitor for corruptions and log them.
--- debian/postinst | 8 ++++++++ debian/prerm | 13 +++++++++++++ debian/rules | 3 ++- 3 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 debian/prerm
debian/postinst +8 -0
@@ -21,5 +21,13 @@ case "${1}" in esac #DEBHELPER# +# +# dh_installsystemd doesn't handle template services even if we supply a +# default instance, so we'll install it here. +if [ -z "${DPKG_ROOT:-}" ] && [ -d /run/systemd/system ] ; then + if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then + /bin/systemctl enable [email protected] || true + fi +fi exit 0
debian/prerm new file +13 -0
@@ -0,0 +1,13 @@ +#!/bin/sh + +set -e + +# dh_installsystemd doesn't handle template services even if we supply a +# default instance, so we'll install it here. +if [ -z "${DPKG_ROOT:-}" ] && [ "$1" = remove ] && [ -d /run/systemd/system ] ; then + /bin/systemctl disable [email protected] || true +fi + +#DEBHELPER# + +exit 0
debian/rules +2 -1
@@ -97,4 +97,5 @@ override_dh_installdocs: dh_installdocs -XCHANGES override_dh_installsystemd: - dh_installsystemd -p xfsprogs --no-restart-after-upgrade --no-stop-on-upgrade system-xfs_scrub.slice xfs_scrub_all.timer + dh_installsystemd -p xfsprogs --no-restart-after-upgrade --no-stop-on-upgrade system-xfs_scrub.slice xfs_scrub_all.timer system-xfs_healer.slice + dh_installsystemd -p xfsprogs --restart-after-upgrade xfs_healer_start.service
Re: [PATCH 28/28] debian: enable xfs_healer on the root filesystem by default
Looks good:
(and I'll see how this will fare on my Debian laptop with the custom built kernel eventually.. :))
PATCH v9 1/2 fstests: test generic file IO error reporting
In reply to Darrick J. WongHi all,
Refactor the iomap file I/O error handling code so that failures are reported in a generic way to fsnotify. Then connect the XFS health reporting to the same fsnotify, and now XFS can notify userspace of problems.
v9: tweak fstests per userspace changes; improve attribution of the
generic fserror test; fix some bugs pointed out by zorro
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=filesystem-error-reporting
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=filesystem-error-reporting
---
Commits in this patchset:
* generic: test fsnotify filesystem error reporting
--- common/rc | 25 ++++++ doc/group-names.txt | 1 src/Makefile | 2 src/fs-monitor.c | 153 ++++++++++++++++++++++++++++++++++ tests/generic/1838 | 214 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/1838.out | 20 ++++ 6 files changed, 414 insertions(+), 1 deletion(-) create mode 100644 src/fs-monitor.c create mode 100755 tests/generic/1838 create mode 100644 tests/generic/1838.out
PATCH 1/1 generic: test fsnotify filesystem error reporting
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
--- common/rc | 25 ++++++ doc/group-names.txt | 1 src/Makefile | 2 src/fs-monitor.c | 153 ++++++++++++++++++++++++++++++++++ tests/generic/1838 | 214 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/1838.out | 20 ++++ 6 files changed, 414 insertions(+), 1 deletion(-) create mode 100644 src/fs-monitor.c create mode 100755 tests/generic/1838 create mode 100644 tests/generic/1838.out
common/rc +25 -0
@@ -3013,6 +3013,11 @@ _require_xfs_io_command() echo $testio | grep -q "Inappropriate ioctl" && \ _notrun "xfs_io $command support is missing" ;; + "healthmon") + test -z "$param" && param="-p" + testio=`$XFS_IO_PROG -c "healthmon $param" $TEST_DIR 2>&1` + param_checked="$param" + ;; "label") testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1` ;;
@@ -6149,6 +6154,26 @@ _require_max_file_range_blocks() esac } +_require_fanotify_ioerrors() +{ + local x + + case "$FSTYP" in + xfs) + # added as a part of xfs health monitoring + _require_xfs_io_command healthmon + return 0 + ;; + ext4) + # added at the same time as err_report_sec + x="$(_get_fs_sysfs_attr $TEST_DEV err_report_sec)" + test -n "$x" && return 0 + ;; + esac + + _notrun "$FSTYP does not support fanotify ioerrors" +} + ################################################################################ # make sure this script returns success /bin/true
doc/group-names.txt +1 -0
@@ -117,6 +117,7 @@ samefs overlayfs when all layers are on the same fs scrub filesystem metadata scrubbers seed btrfs seeded filesystems seek llseek functionality +selfhealing self healing filesystem code selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystem
src/Makefile +1 -1
@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ - rw_hint + rw_hint fs-monitor EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \
src/fs-monitor.c new file +153 -0
153 lines not on this page · raw message
tests/generic/1838 new file +214 -0
214 lines not on this page · raw message
tests/generic/1838.out new file +20 -0
@@ -0,0 +1,20 @@ +QA output created by 1838 +Try buffered read +pread: Input/output error +Try directio read +pread: Input/output error +Try directio write +pwrite: Input/output error +Try buffered write +fsync: Input/output error +Fix device +Kill fsnotify + Generic Error Record: len=XXX + error: 5 +FAN_FS_ERROR (len=XXX) +Remount and restart fsnotify +Try buffered read again +Try directio read again +Try directio write again +Try buffered write again +Kill fsnotify again
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Signed-off-by "Darrick J. Wong" <[email protected]>
Feel free to add:
Quoted 487 lines
--- common/rc | 25 ++++++ doc/group-names.txt | 1 src/Makefile | 2 src/fs-monitor.c | 153 ++++++++++++++++++++++++++++++++++ tests/generic/1838 | 214 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/1838.out | 20 ++++ 6 files changed, 414 insertions(+), 1 deletion(-) create mode 100644 src/fs-monitor.c create mode 100755 tests/generic/1838 create mode 100644 tests/generic/1838.outcommon/rc +25
-0@@ -3013,6 +3013,11 @@ _require_xfs_io_command() echo $testio | grep -q "Inappropriate ioctl" && \ _notrun "xfs_io $command support is missing" ;; + "healthmon") + test -z "$param" && param="-p" + testio=`$XFS_IO_PROG -c "healthmon $param" $TEST_DIR 2>&1` + param_checked="$param" + ;; "label") testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1` ;;@@ -6149,6 +6154,26 @@ _require_max_file_range_blocks() esac } +_require_fanotify_ioerrors() +{ + local x + + case "$FSTYP" in + xfs) + # added as a part of xfs health monitoring + _require_xfs_io_command healthmon + return 0 + ;; + ext4) + # added at the same time as err_report_sec + x="$(_get_fs_sysfs_attr $TEST_DEV err_report_sec)" + test -n "$x" && return 0 + ;; + esac + + _notrun "$FSTYP does not support fanotify ioerrors" +} + ################################################################################ # make sure this script returns success /bin/truedoc/group-names.txt +1
-0@@ -117,6 +117,7 @@ samefs overlayfs when all layers are on the same fs scrub filesystem metadata scrubbers seed btrfs seeded filesystems seek llseek functionality +selfhealing self healing filesystem code selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystemsrc/Makefile +1
-1@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ - rw_hint + rw_hint fs-monitor EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \src/fs-monitor.c new file +153
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright 2021, Collabora Ltd. +* Copied from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/samples/fanotify/fs-monitor.c +*/ + +#include <errno.h> +#include <err.h> +#include <stdlib.h> +#include <stdio.h> +#include <fcntl.h> +#include <sys/fanotify.h> +#include <sys/types.h> +#include <unistd.h> + +#ifndef FAN_FS_ERROR +#define FAN_FS_ERROR 0x00008000 +#define FAN_EVENT_INFO_TYPE_ERROR 5 + +struct fanotify_event_info_error { +struct fanotify_event_info_header hdr; +__s32 error; +__u32 error_count; +}; +#endif + +#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++) +printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]); +else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type); + +} + +static void handle_notifications (char *buffer,int len) +{ +struct fanotify_event_metadata *event = + (struct fanotify_event_metadata *) buffer; +struct fanotify_event_info_header *info; +struct fanotify_event_info_error *err; +struct fanotify_event_info_fid *fid; +int off; + +for (;FAN_EVENT_OK (event, len); event =FAN_EVENT_NEXT (event, len)) { + +if (event->mask !=FAN_FS_ERROR ) { +printf ("unexpected FAN MARK: %llx\n" , + (unsigned long long )event->mask); + goto next_event; + } + +if (event->fd !=FAN_NOFD ) { +printf ("Unexpected fd (!= FAN_NOFD)\n" ); + goto next_event; + } + +printf ("FAN_FS_ERROR (len=%d)\n" , event->event_len); + +for (off =sizeof (*event) ; off < event->event_len; + off += info->len) { + info = (struct fanotify_event_info_header *) + ((char *) event + off); + +switch (info->info_type) { +case FAN_EVENT_INFO_TYPE_ERROR : + err = (struct fanotify_event_info_error *) info; + +printf ("\tGeneric Error Record: len=%d\n" , + err->hdr.len); +printf ("\terror: %d\n" , err->error); +printf ("\terror_count: %d\n" , err->error_count); +break ; + +case FAN_EVENT_INFO_TYPE_FID : + fid = (struct fanotify_event_info_fid *) info; + + printf("\tfsid: %x%x\n", +#if defined(__GLIBC__) + fid->fsid.val[0], fid->fsid.val[1]); +#else + fid->fsid.__val[0], fid->fsid.__val[1]); +#endif +print_fh ((struct file_handle *) &fid->handle); +break ; + +default : +printf ("\tUnknown info type=%d len=%d:\n" , + info->info_type, info->len); + } + } +next_event: +printf ("---\n\n" ); +fflush (stdout); + } +} + +int main (int argc,char **argv) +{ +int fd; + +char buffer[BUFSIZ ]; + +if (argc <2 ) { +printf ("Missing path argument\n" ); +return 1 ; + } + + fd =fanotify_init (FAN_CLASS_NOTIF |FAN_REPORT_FID ,O_RDONLY ); +if (fd <0 ) { +perror ("fanotify_init" ); +errx (1 ,"fanotify_init" ); + } + +if (fanotify_mark (fd,FAN_MARK_ADD |FAN_MARK_FILESYSTEM , +FAN_FS_ERROR ,AT_FDCWD , argv[1 ])) { +perror ("fanotify_mark" ); +errx (1 ,"fanotify_mark" ); + } + +printf ("fanotify active\n" ); +fflush (stdout); + +while (1 ) { +int n =read (fd, buffer,BUFSIZ ); + +if (n <0 ) +errx (1 ,"read" ); + +handle_notifications (buffer, n); + } + +return 0 ; +}tests/generic/1838 new file +214
-0@@ -0,0 +1,214 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1838 +# +# Check that fsnotify can report file IO errors. + +. ./common/preamble +_begin_fstest auto quick eio selfhealing + +# Override the default cleanup function. +_cleanup() +{ + cd / + test -n "$fsmonitor_pid" && kill -TERM $fsmonitor_pid + rm -f $tmp.* + _dmerror_cleanup +} + +# Import common functions. +. ./common/fuzzy +. ./common/filter +. ./common/dmerror +. ./common/systemd + +_require_scratch +_require_dm_target error +_require_test_program fs-monitor +_require_xfs_io_command "fiemap" +_require_odirect +_require_fanotify_ioerrors + +# no out of place writes +test "$FSTYP" = "xfs" && _require_no_xfs_always_cow + +# fsnotify only gives us a file handle, the error number, and the number of +# times it was seen in between event deliveries. The handle is mostly useless +# since we have no generic way to map that to a file path. Therefore we can +# only coalesce all the I/O errors into one report. +filter_fsnotify_errors() { + _filter_scratch | \ + grep -E '(FAN_FS_ERROR|Generic Error Record|error: 5)' | \ + sed -e "s/len=[0-9]*/len=XXX/g" | \ + sort | \ + uniq +} + +_scratch_mkfs >> $seqres.full + +# +# The dm-error map added by this test doesn't work on zoned devices because +# table sizes need to be aligned to the zone size, and even for zoned on +# conventional this test will get confused because of the internal RT device. +# +# That check requires a mounted file system, so do a dummy mount before setting +# up DM. +# +_scratch_mount +test $FSTYP = xfs && _require_xfs_scratch_non_zoned +_scratch_unmount + +_dmerror_init +_dmerror_mount >> $seqres.full 2>&1 + +test $FSTYP = xfs && _xfs_force_bdev data $SCRATCH_MNT + +# Write a file with 4 file blocks worth of data, figure out the LBA to target +victim=$SCRATCH_MNT/a +file_blksz=$(_get_file_block_size $SCRATCH_MNT) +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full + +awk_len_prog='{print $4}' +bmap_str="$($XFS_IO_PROG -c "fiemap -v" $victim | grep "^[[:space:]]*0:")" +echo "$bmap_str" >> $seqres.full + +phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')" +len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")" + +fs_blksz=$(_get_block_size $SCRATCH_MNT) +echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full +kernel_sectors_per_fs_block=$((fs_blksz / 512)) + +# Did we get at least 4 fs blocks worth of extent? +min_len_sectors=$(( 4 * kernel_sectors_per_fs_block )) +test "$len" -lt $min_len_sectors && \ + _fail "could not format a long enough extent on an empty fs??" + +phys_start=$(echo "$phys" | sed -e 's/\.\..*//g') + +echo "$phys:$len:$fs_blksz:$phys_start" >> $seqres.full +echo "victim file:" >> $seqres.full +od -tx1 -Ad -c $victim >> $seqres.full + +# Set the dmerror table so that all IO will pass through. +_dmerror_reset_table + +cat >> $seqres.full << ENDL +dmerror before: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# All sector numbers that we feed to the kernel must be in units of 512b, but +# they also must be aligned to the device's logical block size. +logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` +kernel_sectors_per_device_lba=$((logical_block_size / 512)) + +# Mark as bad one of the device LBAs in the middle of the extent. Target the +# second LBA of the third block of the four-block file extent that we allocated +# earlier, but without overflowing into the fourth file block. +bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) )) +bad_len=$kernel_sectors_per_device_lba +if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then + bad_sector=$((bad_sector + kernel_sectors_per_device_lba)) +fi +if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then + echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size" +fi + +# Remount to flush the page cache, start fsnotify, and make the LBA bad +_dmerror_unmount +_dmerror_mount + +$here/src/fs-monitor $SCRATCH_MNT > $tmp.fsmonitor & +fsmonitor_pid=$! +sleep 1 + +_dmerror_mark_range_bad $bad_sector $bad_len + +cat >> $seqres.full << ENDL +dmerror after marking bad: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +_dmerror_load_error_table + +# See if buffered reads pick it up +echo "Try buffered read" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Now mark the bad range good so that unmount won't fail due to IO errors. +echo "Fix device" +_dmerror_mark_range_good $bad_sector $bad_len +_dmerror_load_error_table + +cat >> $seqres.full << ENDL +dmerror after marking good: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# Unmount filesystem to start fresh +echo "Kill fsnotify" +_dmerror_unmount +sleep 1 +kill -TERM $fsmonitor_pid +unset fsmonitor_pid +echo fsnotify log >> $seqres.full +cat $tmp.fsmonitor >> $seqres.full +cat $tmp.fsmonitor | filter_fsnotify_errors + +# Start fsnotify again so that can verify that the errors don't persist after +# we flip back to the good dm table. +echo "Remount and restart fsnotify" +_dmerror_mount +$here/src/fs-monitor $SCRATCH_MNT > $tmp.fsmonitor & +fsmonitor_pid=$! +sleep 1 + +# See if buffered reads pick it up +echo "Try buffered read again" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read again" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write again" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write again" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Unmount fs and kill fsnotify, then wait for it to finish +echo "Kill fsnotify again" +_dmerror_unmount +sleep 1 +kill -TERM $fsmonitor_pid +unset fsmonitor_pid +cat $tmp.fsmonitor >> $seqres.full +cat $tmp.fsmonitor | filter_fsnotify_errors + +# success, all done +status=0 +exittests/generic/1838.out new file +20
-0@@ -0,0 +1,20 @@ +QA output created by 1838 +Try buffered read +pread: Input/output error +Try directio read +pread: Input/output error +Try directio write +pwrite: Input/output error +Try buffered write +fsync: Input/output error +Fix device +Kill fsnotify + Generic Error Record: len=XXX + error: 5 +FAN_FS_ERROR (len=XXX) +Remount and restart fsnotify +Try buffered read again +Try directio read again +Try directio write again +Try buffered write again +Kill fsnotify again
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
In reply to Darrick J. WongQuoted 10 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Signed-off-by "Darrick J. Wong" <[email protected]>--- common/rc | 25 ++++++ doc/group-names.txt | 1 src/Makefile | 2 src/fs-monitor.c | 153 ++++++++++++++++++++++++++++++++++
I'll add "src/fs-monitor" into .gitignore file when I merge this patch. Others look good to me.
Quoted 482 lines
tests/generic/1838 | 214 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/1838.out | 20 ++++ 6 files changed, 414 insertions(+), 1 deletion(-) create mode 100644 src/fs-monitor.c create mode 100755 tests/generic/1838 create mode 100644 tests/generic/1838.outcommon/rc +25
-0@@ -3013,6 +3013,11 @@ _require_xfs_io_command() echo $testio | grep -q "Inappropriate ioctl" && \ _notrun "xfs_io $command support is missing" ;; + "healthmon") + test -z "$param" && param="-p" + testio=`$XFS_IO_PROG -c "healthmon $param" $TEST_DIR 2>&1` + param_checked="$param" + ;; "label") testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1` ;;@@ -6149,6 +6154,26 @@ _require_max_file_range_blocks() esac } +_require_fanotify_ioerrors() +{ + local x + + case "$FSTYP" in + xfs) + # added as a part of xfs health monitoring + _require_xfs_io_command healthmon + return 0 + ;; + ext4) + # added at the same time as err_report_sec + x="$(_get_fs_sysfs_attr $TEST_DEV err_report_sec)" + test -n "$x" && return 0 + ;; + esac + + _notrun "$FSTYP does not support fanotify ioerrors" +} + ################################################################################ # make sure this script returns success /bin/truedoc/group-names.txt +1
-0@@ -117,6 +117,7 @@ samefs overlayfs when all layers are on the same fs scrub filesystem metadata scrubbers seed btrfs seeded filesystems seek llseek functionality +selfhealing self healing filesystem code selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystemsrc/Makefile +1
-1@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ - rw_hint + rw_hint fs-monitor EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \src/fs-monitor.c new file +153
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright 2021, Collabora Ltd. +* Copied from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/samples/fanotify/fs-monitor.c +*/ + +#include <errno.h> +#include <err.h> +#include <stdlib.h> +#include <stdio.h> +#include <fcntl.h> +#include <sys/fanotify.h> +#include <sys/types.h> +#include <unistd.h> + +#ifndef FAN_FS_ERROR +#define FAN_FS_ERROR 0x00008000 +#define FAN_EVENT_INFO_TYPE_ERROR 5 + +struct fanotify_event_info_error { +struct fanotify_event_info_header hdr; +__s32 error; +__u32 error_count; +}; +#endif + +#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++) +printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]); +else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type); + +} + +static void handle_notifications (char *buffer,int len) +{ +struct fanotify_event_metadata *event = + (struct fanotify_event_metadata *) buffer; +struct fanotify_event_info_header *info; +struct fanotify_event_info_error *err; +struct fanotify_event_info_fid *fid; +int off; + +for (;FAN_EVENT_OK (event, len); event =FAN_EVENT_NEXT (event, len)) { + +if (event->mask !=FAN_FS_ERROR ) { +printf ("unexpected FAN MARK: %llx\n" , + (unsigned long long )event->mask); + goto next_event; + } + +if (event->fd !=FAN_NOFD ) { +printf ("Unexpected fd (!= FAN_NOFD)\n" ); + goto next_event; + } + +printf ("FAN_FS_ERROR (len=%d)\n" , event->event_len); + +for (off =sizeof (*event) ; off < event->event_len; + off += info->len) { + info = (struct fanotify_event_info_header *) + ((char *) event + off); + +switch (info->info_type) { +case FAN_EVENT_INFO_TYPE_ERROR : + err = (struct fanotify_event_info_error *) info; + +printf ("\tGeneric Error Record: len=%d\n" , + err->hdr.len); +printf ("\terror: %d\n" , err->error); +printf ("\terror_count: %d\n" , err->error_count); +break ; + +case FAN_EVENT_INFO_TYPE_FID : + fid = (struct fanotify_event_info_fid *) info; + + printf("\tfsid: %x%x\n", +#if defined(__GLIBC__) + fid->fsid.val[0], fid->fsid.val[1]); +#else + fid->fsid.__val[0], fid->fsid.__val[1]); +#endif +print_fh ((struct file_handle *) &fid->handle); +break ; + +default : +printf ("\tUnknown info type=%d len=%d:\n" , + info->info_type, info->len); + } + } +next_event: +printf ("---\n\n" ); +fflush (stdout); + } +} + +int main (int argc,char **argv) +{ +int fd; + +char buffer[BUFSIZ ]; + +if (argc <2 ) { +printf ("Missing path argument\n" ); +return 1 ; + } + + fd =fanotify_init (FAN_CLASS_NOTIF |FAN_REPORT_FID ,O_RDONLY ); +if (fd <0 ) { +perror ("fanotify_init" ); +errx (1 ,"fanotify_init" ); + } + +if (fanotify_mark (fd,FAN_MARK_ADD |FAN_MARK_FILESYSTEM , +FAN_FS_ERROR ,AT_FDCWD , argv[1 ])) { +perror ("fanotify_mark" ); +errx (1 ,"fanotify_mark" ); + } + +printf ("fanotify active\n" ); +fflush (stdout); + +while (1 ) { +int n =read (fd, buffer,BUFSIZ ); + +if (n <0 ) +errx (1 ,"read" ); + +handle_notifications (buffer, n); + } + +return 0 ; +}tests/generic/1838 new file +214
-0@@ -0,0 +1,214 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1838 +# +# Check that fsnotify can report file IO errors. + +. ./common/preamble +_begin_fstest auto quick eio selfhealing + +# Override the default cleanup function. +_cleanup() +{ + cd / + test -n "$fsmonitor_pid" && kill -TERM $fsmonitor_pid + rm -f $tmp.* + _dmerror_cleanup +} + +# Import common functions. +. ./common/fuzzy +. ./common/filter +. ./common/dmerror +. ./common/systemd + +_require_scratch +_require_dm_target error +_require_test_program fs-monitor +_require_xfs_io_command "fiemap" +_require_odirect +_require_fanotify_ioerrors + +# no out of place writes +test "$FSTYP" = "xfs" && _require_no_xfs_always_cow + +# fsnotify only gives us a file handle, the error number, and the number of +# times it was seen in between event deliveries. The handle is mostly useless +# since we have no generic way to map that to a file path. Therefore we can +# only coalesce all the I/O errors into one report. +filter_fsnotify_errors() { + _filter_scratch | \ + grep -E '(FAN_FS_ERROR|Generic Error Record|error: 5)' | \ + sed -e "s/len=[0-9]*/len=XXX/g" | \ + sort | \ + uniq +} + +_scratch_mkfs >> $seqres.full + +# +# The dm-error map added by this test doesn't work on zoned devices because +# table sizes need to be aligned to the zone size, and even for zoned on +# conventional this test will get confused because of the internal RT device. +# +# That check requires a mounted file system, so do a dummy mount before setting +# up DM. +# +_scratch_mount +test $FSTYP = xfs && _require_xfs_scratch_non_zoned +_scratch_unmount + +_dmerror_init +_dmerror_mount >> $seqres.full 2>&1 + +test $FSTYP = xfs && _xfs_force_bdev data $SCRATCH_MNT + +# Write a file with 4 file blocks worth of data, figure out the LBA to target +victim=$SCRATCH_MNT/a +file_blksz=$(_get_file_block_size $SCRATCH_MNT) +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full + +awk_len_prog='{print $4}' +bmap_str="$($XFS_IO_PROG -c "fiemap -v" $victim | grep "^[[:space:]]*0:")" +echo "$bmap_str" >> $seqres.full + +phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')" +len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")" + +fs_blksz=$(_get_block_size $SCRATCH_MNT) +echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full +kernel_sectors_per_fs_block=$((fs_blksz / 512)) + +# Did we get at least 4 fs blocks worth of extent? +min_len_sectors=$(( 4 * kernel_sectors_per_fs_block )) +test "$len" -lt $min_len_sectors && \ + _fail "could not format a long enough extent on an empty fs??" + +phys_start=$(echo "$phys" | sed -e 's/\.\..*//g') + +echo "$phys:$len:$fs_blksz:$phys_start" >> $seqres.full +echo "victim file:" >> $seqres.full +od -tx1 -Ad -c $victim >> $seqres.full + +# Set the dmerror table so that all IO will pass through. +_dmerror_reset_table + +cat >> $seqres.full << ENDL +dmerror before: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# All sector numbers that we feed to the kernel must be in units of 512b, but +# they also must be aligned to the device's logical block size. +logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` +kernel_sectors_per_device_lba=$((logical_block_size / 512)) + +# Mark as bad one of the device LBAs in the middle of the extent. Target the +# second LBA of the third block of the four-block file extent that we allocated +# earlier, but without overflowing into the fourth file block. +bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) )) +bad_len=$kernel_sectors_per_device_lba +if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then + bad_sector=$((bad_sector + kernel_sectors_per_device_lba)) +fi +if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then + echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size" +fi + +# Remount to flush the page cache, start fsnotify, and make the LBA bad +_dmerror_unmount +_dmerror_mount + +$here/src/fs-monitor $SCRATCH_MNT > $tmp.fsmonitor & +fsmonitor_pid=$! +sleep 1 + +_dmerror_mark_range_bad $bad_sector $bad_len + +cat >> $seqres.full << ENDL +dmerror after marking bad: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +_dmerror_load_error_table + +# See if buffered reads pick it up +echo "Try buffered read" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Now mark the bad range good so that unmount won't fail due to IO errors. +echo "Fix device" +_dmerror_mark_range_good $bad_sector $bad_len +_dmerror_load_error_table + +cat >> $seqres.full << ENDL +dmerror after marking good: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# Unmount filesystem to start fresh +echo "Kill fsnotify" +_dmerror_unmount +sleep 1 +kill -TERM $fsmonitor_pid +unset fsmonitor_pid +echo fsnotify log >> $seqres.full +cat $tmp.fsmonitor >> $seqres.full +cat $tmp.fsmonitor | filter_fsnotify_errors + +# Start fsnotify again so that can verify that the errors don't persist after +# we flip back to the good dm table. +echo "Remount and restart fsnotify" +_dmerror_mount +$here/src/fs-monitor $SCRATCH_MNT > $tmp.fsmonitor & +fsmonitor_pid=$! +sleep 1 + +# See if buffered reads pick it up +echo "Try buffered read again" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read again" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write again" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write again" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Unmount fs and kill fsnotify, then wait for it to finish +echo "Kill fsnotify again" +_dmerror_unmount +sleep 1 +kill -TERM $fsmonitor_pid +unset fsmonitor_pid +cat $tmp.fsmonitor >> $seqres.full +cat $tmp.fsmonitor | filter_fsnotify_errors + +# success, all done +status=0 +exittests/generic/1838.out new file +20
-0@@ -0,0 +1,20 @@ +QA output created by 1838 +Try buffered read +pread: Input/output error +Try directio read +pread: Input/output error +Try directio write +pwrite: Input/output error +Try buffered write +fsync: Input/output error +Fix device +Kill fsnotify + Generic Error Record: len=XXX + error: 5 +FAN_FS_ERROR (len=XXX) +Remount and restart fsnotify +Try buffered read again +Try directio read again +Try directio write again +Try buffered write again +Kill fsnotify again
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
Quoted 12 lines · Zorro Lang
Quoted 10 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Signed-off-by "Darrick J. Wong" <[email protected]>--- common/rc | 25 ++++++ doc/group-names.txt | 1 src/Makefile | 2 src/fs-monitor.c | 153 ++++++++++++++++++++++++++++++++++
Quoted 2 lines
I'll add "src/fs-monitor" into .gitignore file when I merge this patch. Others look good to me.
Oops, thanks for fixing that for me!
--D
Quoted 486 lines
Reviewed-by Zorro Lang <[email protected]>Quoted 482 lines
tests/generic/1838 | 214 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/1838.out | 20 ++++ 6 files changed, 414 insertions(+), 1 deletion(-) create mode 100644 src/fs-monitor.c create mode 100755 tests/generic/1838 create mode 100644 tests/generic/1838.outcommon/rc +25
-0@@ -3013,6 +3013,11 @@ _require_xfs_io_command() echo $testio | grep -q "Inappropriate ioctl" && \ _notrun "xfs_io $command support is missing" ;; + "healthmon") + test -z "$param" && param="-p" + testio=`$XFS_IO_PROG -c "healthmon $param" $TEST_DIR 2>&1` + param_checked="$param" + ;; "label") testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1` ;;@@ -6149,6 +6154,26 @@ _require_max_file_range_blocks() esac } +_require_fanotify_ioerrors() +{ + local x + + case "$FSTYP" in + xfs) + # added as a part of xfs health monitoring + _require_xfs_io_command healthmon + return 0 + ;; + ext4) + # added at the same time as err_report_sec + x="$(_get_fs_sysfs_attr $TEST_DEV err_report_sec)" + test -n "$x" && return 0 + ;; + esac + + _notrun "$FSTYP does not support fanotify ioerrors" +} + ################################################################################ # make sure this script returns success /bin/truedoc/group-names.txt +1
-0@@ -117,6 +117,7 @@ samefs overlayfs when all layers are on the same fs scrub filesystem metadata scrubbers seed btrfs seeded filesystems seek llseek functionality +selfhealing self healing filesystem code selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystemsrc/Makefile +1
-1@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ - rw_hint + rw_hint fs-monitor EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \src/fs-monitor.c new file +153
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright 2021, Collabora Ltd. +* Copied from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/samples/fanotify/fs-monitor.c +*/ + +#include <errno.h> +#include <err.h> +#include <stdlib.h> +#include <stdio.h> +#include <fcntl.h> +#include <sys/fanotify.h> +#include <sys/types.h> +#include <unistd.h> + +#ifndef FAN_FS_ERROR +#define FAN_FS_ERROR 0x00008000 +#define FAN_EVENT_INFO_TYPE_ERROR 5 + +struct fanotify_event_info_error { +struct fanotify_event_info_header hdr; +__s32 error; +__u32 error_count; +}; +#endif + +#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++) +printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]); +else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type); + +} + +static void handle_notifications (char *buffer,int len) +{ +struct fanotify_event_metadata *event = + (struct fanotify_event_metadata *) buffer; +struct fanotify_event_info_header *info; +struct fanotify_event_info_error *err; +struct fanotify_event_info_fid *fid; +int off; + +for (;FAN_EVENT_OK (event, len); event =FAN_EVENT_NEXT (event, len)) { + +if (event->mask !=FAN_FS_ERROR ) { +printf ("unexpected FAN MARK: %llx\n" , + (unsigned long long )event->mask); + goto next_event; + } + +if (event->fd !=FAN_NOFD ) { +printf ("Unexpected fd (!= FAN_NOFD)\n" ); + goto next_event; + } + +printf ("FAN_FS_ERROR (len=%d)\n" , event->event_len); + +for (off =sizeof (*event) ; off < event->event_len; + off += info->len) { + info = (struct fanotify_event_info_header *) + ((char *) event + off); + +switch (info->info_type) { +case FAN_EVENT_INFO_TYPE_ERROR : + err = (struct fanotify_event_info_error *) info; + +printf ("\tGeneric Error Record: len=%d\n" , + err->hdr.len); +printf ("\terror: %d\n" , err->error); +printf ("\terror_count: %d\n" , err->error_count); +break ; + +case FAN_EVENT_INFO_TYPE_FID : + fid = (struct fanotify_event_info_fid *) info; + + printf("\tfsid: %x%x\n", +#if defined(__GLIBC__) + fid->fsid.val[0], fid->fsid.val[1]); +#else + fid->fsid.__val[0], fid->fsid.__val[1]); +#endif +print_fh ((struct file_handle *) &fid->handle); +break ; + +default : +printf ("\tUnknown info type=%d len=%d:\n" , + info->info_type, info->len); + } + } +next_event: +printf ("---\n\n" ); +fflush (stdout); + } +} + +int main (int argc,char **argv) +{ +int fd; + +char buffer[BUFSIZ ]; + +if (argc <2 ) { +printf ("Missing path argument\n" ); +return 1 ; + } + + fd =fanotify_init (FAN_CLASS_NOTIF |FAN_REPORT_FID ,O_RDONLY ); +if (fd <0 ) { +perror ("fanotify_init" ); +errx (1 ,"fanotify_init" ); + } + +if (fanotify_mark (fd,FAN_MARK_ADD |FAN_MARK_FILESYSTEM , +FAN_FS_ERROR ,AT_FDCWD , argv[1 ])) { +perror ("fanotify_mark" ); +errx (1 ,"fanotify_mark" ); + } + +printf ("fanotify active\n" ); +fflush (stdout); + +while (1 ) { +int n =read (fd, buffer,BUFSIZ ); + +if (n <0 ) +errx (1 ,"read" ); + +handle_notifications (buffer, n); + } + +return 0 ; +}tests/generic/1838 new file +214
-0@@ -0,0 +1,214 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1838 +# +# Check that fsnotify can report file IO errors. + +. ./common/preamble +_begin_fstest auto quick eio selfhealing + +# Override the default cleanup function. +_cleanup() +{ + cd / + test -n "$fsmonitor_pid" && kill -TERM $fsmonitor_pid + rm -f $tmp.* + _dmerror_cleanup +} + +# Import common functions. +. ./common/fuzzy +. ./common/filter +. ./common/dmerror +. ./common/systemd + +_require_scratch +_require_dm_target error +_require_test_program fs-monitor +_require_xfs_io_command "fiemap" +_require_odirect +_require_fanotify_ioerrors + +# no out of place writes +test "$FSTYP" = "xfs" && _require_no_xfs_always_cow + +# fsnotify only gives us a file handle, the error number, and the number of +# times it was seen in between event deliveries. The handle is mostly useless +# since we have no generic way to map that to a file path. Therefore we can +# only coalesce all the I/O errors into one report. +filter_fsnotify_errors() { + _filter_scratch | \ + grep -E '(FAN_FS_ERROR|Generic Error Record|error: 5)' | \ + sed -e "s/len=[0-9]*/len=XXX/g" | \ + sort | \ + uniq +} + +_scratch_mkfs >> $seqres.full + +# +# The dm-error map added by this test doesn't work on zoned devices because +# table sizes need to be aligned to the zone size, and even for zoned on +# conventional this test will get confused because of the internal RT device. +# +# That check requires a mounted file system, so do a dummy mount before setting +# up DM. +# +_scratch_mount +test $FSTYP = xfs && _require_xfs_scratch_non_zoned +_scratch_unmount + +_dmerror_init +_dmerror_mount >> $seqres.full 2>&1 + +test $FSTYP = xfs && _xfs_force_bdev data $SCRATCH_MNT + +# Write a file with 4 file blocks worth of data, figure out the LBA to target +victim=$SCRATCH_MNT/a +file_blksz=$(_get_file_block_size $SCRATCH_MNT) +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full + +awk_len_prog='{print $4}' +bmap_str="$($XFS_IO_PROG -c "fiemap -v" $victim | grep "^[[:space:]]*0:")" +echo "$bmap_str" >> $seqres.full + +phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')" +len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")" + +fs_blksz=$(_get_block_size $SCRATCH_MNT) +echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full +kernel_sectors_per_fs_block=$((fs_blksz / 512)) + +# Did we get at least 4 fs blocks worth of extent? +min_len_sectors=$(( 4 * kernel_sectors_per_fs_block )) +test "$len" -lt $min_len_sectors && \ + _fail "could not format a long enough extent on an empty fs??" + +phys_start=$(echo "$phys" | sed -e 's/\.\..*//g') + +echo "$phys:$len:$fs_blksz:$phys_start" >> $seqres.full +echo "victim file:" >> $seqres.full +od -tx1 -Ad -c $victim >> $seqres.full + +# Set the dmerror table so that all IO will pass through. +_dmerror_reset_table + +cat >> $seqres.full << ENDL +dmerror before: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# All sector numbers that we feed to the kernel must be in units of 512b, but +# they also must be aligned to the device's logical block size. +logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` +kernel_sectors_per_device_lba=$((logical_block_size / 512)) + +# Mark as bad one of the device LBAs in the middle of the extent. Target the +# second LBA of the third block of the four-block file extent that we allocated +# earlier, but without overflowing into the fourth file block. +bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) )) +bad_len=$kernel_sectors_per_device_lba +if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then + bad_sector=$((bad_sector + kernel_sectors_per_device_lba)) +fi +if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then + echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size" +fi + +# Remount to flush the page cache, start fsnotify, and make the LBA bad +_dmerror_unmount +_dmerror_mount + +$here/src/fs-monitor $SCRATCH_MNT > $tmp.fsmonitor & +fsmonitor_pid=$! +sleep 1 + +_dmerror_mark_range_bad $bad_sector $bad_len + +cat >> $seqres.full << ENDL +dmerror after marking bad: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +_dmerror_load_error_table + +# See if buffered reads pick it up +echo "Try buffered read" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Now mark the bad range good so that unmount won't fail due to IO errors. +echo "Fix device" +_dmerror_mark_range_good $bad_sector $bad_len +_dmerror_load_error_table + +cat >> $seqres.full << ENDL +dmerror after marking good: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# Unmount filesystem to start fresh +echo "Kill fsnotify" +_dmerror_unmount +sleep 1 +kill -TERM $fsmonitor_pid +unset fsmonitor_pid +echo fsnotify log >> $seqres.full +cat $tmp.fsmonitor >> $seqres.full +cat $tmp.fsmonitor | filter_fsnotify_errors + +# Start fsnotify again so that can verify that the errors don't persist after +# we flip back to the good dm table. +echo "Remount and restart fsnotify" +_dmerror_mount +$here/src/fs-monitor $SCRATCH_MNT > $tmp.fsmonitor & +fsmonitor_pid=$! +sleep 1 + +# See if buffered reads pick it up +echo "Try buffered read again" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read again" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write again" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write again" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Unmount fs and kill fsnotify, then wait for it to finish +echo "Kill fsnotify again" +_dmerror_unmount +sleep 1 +kill -TERM $fsmonitor_pid +unset fsmonitor_pid +cat $tmp.fsmonitor >> $seqres.full +cat $tmp.fsmonitor | filter_fsnotify_errors + +# success, all done +status=0 +exittests/generic/1838.out new file +20
-0@@ -0,0 +1,20 @@ +QA output created by 1838 +Try buffered read +pread: Input/output error +Try directio read +pread: Input/output error +Try directio write +pwrite: Input/output error +Try buffered write +fsync: Input/output error +Fix device +Kill fsnotify + Generic Error Record: len=XXX + error: 5 +FAN_FS_ERROR (len=XXX) +Remount and restart fsnotify +Try buffered read again +Try directio read again +Try directio write again +Try buffered write again +Kill fsnotify again
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
In reply to Darrick J. WongQuoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Still would be helpful to explain what is being tested and how and the source of the test tool.
Quoted 15 lines
src/fs-monitor.c new file +15
-0+#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++)
src/fs-monitor.c line 51
Quoted 10 lines
src/fs-monitor.c new file +10
-0+printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]); +else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type);
Isn't this always going to print unknown for normal xfs mounts without inode32?
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Still would be helpful to explain what is being tested and how and the source of the test tool.
"Use dmerror to inject a media error for a file's storage, then run some IO and make sure that fanotify reports the IO errors."
(I'm not sure what the process is for updating commit messages once something's in patches-in-queue...)
Quoted 21 lines
Quoted 21 lines
src/fs-monitor.c new file +21
-0+#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++) +printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]);
Quoted 8 lines
src/fs-monitor.c line 51
Quoted 4 lines
src/fs-monitor.c new file +4
-0+else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type);Isn't this always going to print unknown for normal xfs mounts without inode32?
Yes, though generic/791 filters out the file handle and (TBH) I don't really want people getting ideas about cracking file handles. Probably we should just eliminate this whole part of the function, but that would make future forklift upgrades from the kernel harder.
--D
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
Quoted 6 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Quoted 8 lines
Quoted 2 lines
Still would be helpful to explain what is being tested and how and the source of the test tool.
"Use dmerror to inject a media error for a file's storage, then run some IO and make sure that fanotify reports the IO errors."
(I'm not sure what the process is for updating commit messages once something's in patches-in-queue...)
patches-in-queue is a scratch branch, I always re-push(-f) to this branch. So please feel free to tell me what do you want to update in this patch (I'm going to make the next fstests release soon:).
Thanks,
Zorro
Quoted 38 lines
Quoted 22 lines
Quoted 22 lines
src/fs-monitor.c new file +21
-0+#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++) +printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]);Quoted 8 lines
src/fs-monitor.c line 51
Quoted 4 lines
src/fs-monitor.c new file +4
-0+else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type);Isn't this always going to print unknown for normal xfs mounts without inode32?
Yes, though generic/791 filters out the file handle and (TBH) I don't really want people getting ideas about cracking file handles. Probably we should just eliminate this whole part of the function, but that would make future forklift upgrades from the kernel harder.
--D
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
Quoted 13 lines · Zorro Lang
Quoted 12 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Still would be helpful to explain what is being tested and how and the source of the test tool.
"Use dmerror to inject a media error for a file's storage, then run some IO and make sure that fanotify reports the IO errors."
Quoted 6 lines
Quoted 2 lines
(I'm not sure what the process is for updating commit messages once something's in patches-in-queue...)
patches-in-queue is a scratch branch, I always re-push(-f) to this branch. So please feel free to tell me what do you want to update in this patch (I'm going to make the next fstests release soon:).
Just the two changes I've mentioned so far in the threads, please.
I've more fixes to make to these tests, so I'll send those patches shortly.
--D
Quoted 42 lines
Thanks,
ZorroQuoted 38 lines
Quoted 22 lines
Quoted 22 lines
src/fs-monitor.c new file +21
-0+#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++) +printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]);Quoted 8 lines
src/fs-monitor.c line 51
Quoted 4 lines
src/fs-monitor.c new file +4
-0+else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type);Isn't this always going to print unknown for normal xfs mounts without inode32?
Yes, though generic/791 filters out the file handle and (TBH) I don't really want people getting ideas about cracking file handles. Probably we should just eliminate this whole part of the function, but that would make future forklift upgrades from the kernel harder.
--D
Re: [PATCH 1/1] generic: test fsnotify filesystem error reporting
Quoted 14 lines · Darrick J. Wong
Quoted 13 lines · Zorro Lang
Quoted 12 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Test the fsnotify filesystem error reporting.
Still would be helpful to explain what is being tested and how and the source of the test tool.
"Use dmerror to inject a media error for a file's storage, then run some IO and make sure that fanotify reports the IO errors."
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
(I'm not sure what the process is for updating commit messages once something's in patches-in-queue...)
patches-in-queue is a scratch branch, I always re-push(-f) to this branch. So please feel free to tell me what do you want to update in this patch (I'm going to make the next fstests release soon:).
Just the two changes I've mentioned so far in the threads, please.
Sure, I just changed the commit log of this patch to:
"Use dmerror to inject a media error for a file's storage, then run some IO and make sure that fanotify reports the IO errors."
but what's the 2nd change? Did I miss something? ...
Quoted 44 lines
I've more fixes to make to these tests, so I'll send those patches shortly.
--D
Quoted 38 lines
Thanks,
ZorroQuoted 35 lines
Quoted 22 lines
Quoted 22 lines
src/fs-monitor.c new file +21
-0+#ifndef FILEID_INO32_GEN +#define FILEID_INO32_GEN 1 +#endif + +#ifndef FILEID_INVALID +#define FILEID_INVALID 0xff +#endif + +static void print_fh (struct file_handle *fh) +{ +int i; +uint32_t *h = (uint32_t *) fh->f_handle; + +printf ("\tfh: " ); +for (i =0 ; i < fh->handle_bytes; i++) +printf ("%hhx" , fh->f_handle[i]); +printf ("\n" ); + +printf ("\tdecoded fh: " ); +if (fh->handle_type ==FILEID_INO32_GEN ) +printf ("inode=%u gen=%u\n" , h[0 ], h[1 ]);Quoted 8 lines
src/fs-monitor.c line 51
Quoted 4 lines
src/fs-monitor.c new file +4
-0+else if (fh->handle_type ==FILEID_INVALID && !fh->handle_bytes) +printf ("Type %d (Superblock error)\n" , fh->handle_type); +else +printf ("Type %d (Unknown)\n" , fh->handle_type);Isn't this always going to print unknown for normal xfs mounts without inode32?
Yes, though generic/791 filters out the file handle and (TBH) I don't really want people getting ideas about cracking file handles. Probably we should just eliminate this whole part of the function, but that would make future forklift upgrades from the kernel harder.
... This looks like not a certain decision/change. Is this the 2nd change you want to do? If so, could you please provide what's the specific changes you want to make :)
Thanks,
Zorro
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
--D
PATCH v9 2/2 fstests: autonomous self healing of filesystems
In reply to Darrick J. WongHi all,
This series adds functionality and regression tests for the automated self healing daemon for xfs.
v9: tweak fstests per userspace changes; improve attribution of the
generic fserror test; fix some bugs pointed out by zorro
v8: clean up userspace for merging now that the kernel part is upstream
v7: more cleanups of the media verification ioctl, improve comments, and
reuse the bio
v6: fix pi-breaking bugs, make verify failures trigger health reports
v5: add verify-media ioctl, collapse small helper funcs with only
one caller
v4: drop multiple client support so we can make direct calls into
healthmon instead of chasing pointers and doing indirect calls
v3: drag out of rfc status
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
With a bit of luck, this should all go splendidly. Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=health-monitoring
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=health-monitoring
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=health-monitoring
---
Commits in this patchset:
* xfs: test health monitoring code
* xfs: test for metadata corruption error reporting via healthmon
* xfs: test io error reporting via healthmon
* xfs: set up common code for testing xfs_healer
* xfs: test xfs_healer's event handling
* xfs: test xfs_healer can fix a filesystem
* xfs: test xfs_healer can report file I/O errors
* xfs: test xfs_healer can report file media errors
* xfs: test xfs_healer can report filesystem shutdowns
* xfs: test xfs_healer can initiate full filesystem repairs
* xfs: test xfs_healer can follow mount moves
* xfs: test xfs_healer wont repair the wrong filesystem
* xfs: test xfs_healer background service
* xfs: test xfs_healer startup service
--- common/config | 14 +++ common/module | 11 +++ common/rc | 5 + common/systemd | 39 ++++++++++ common/xfs | 98 ++++++++++++++++++++++++ tests/xfs/1878 | 93 +++++++++++++++++++++++ tests/xfs/1878.out | 10 ++ tests/xfs/1879 | 93 +++++++++++++++++++++++ tests/xfs/1879.out | 8 ++ tests/xfs/1882 | 44 +++++++++++ tests/xfs/1882.out | 2 tests/xfs/1884 | 89 ++++++++++++++++++++++ tests/xfs/1884.out | 2 tests/xfs/1885 | 59 +++++++++++++++ tests/xfs/1885.out | 5 + tests/xfs/1896 | 210 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1896.out | 21 +++++ tests/xfs/1897 | 172 +++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1897.out | 7 ++ tests/xfs/1898 | 37 +++++++++ tests/xfs/1898.out | 4 + tests/xfs/1899 | 108 +++++++++++++++++++++++++++ tests/xfs/1899.out | 3 + tests/xfs/1900 | 115 ++++++++++++++++++++++++++++ tests/xfs/1900.out | 2 tests/xfs/1901 | 137 ++++++++++++++++++++++++++++++++++ tests/xfs/1901.out | 2 tests/xfs/1902 | 152 ++++++++++++++++++++++++++++++++++++++ tests/xfs/1902.out | 2 tests/xfs/1903 | 124 +++++++++++++++++++++++++++++++ tests/xfs/1903.out | 6 + tests/xfs/802 | 4 - 32 files changed, 1676 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/1878 create mode 100644 tests/xfs/1878.out create mode 100755 tests/xfs/1879 create mode 100644 tests/xfs/1879.out create mode 100755 tests/xfs/1882 create mode 100644 tests/xfs/1882.out create mode 100755 tests/xfs/1884 create mode 100644 tests/xfs/1884.out create mode 100755 tests/xfs/1885 create mode 100644 tests/xfs/1885.out create mode 100755 tests/xfs/1896 create mode 100644 tests/xfs/1896.out create mode 100755 tests/xfs/1897 create mode 100755 tests/xfs/1897.out create mode 100755 tests/xfs/1898 create mode 100755 tests/xfs/1898.out create mode 100755 tests/xfs/1899 create mode 100644 tests/xfs/1899.out create mode 100755 tests/xfs/1900 create mode 100755 tests/xfs/1900.out create mode 100755 tests/xfs/1901 create mode 100755 tests/xfs/1901.out create mode 100755 tests/xfs/1902 create mode 100755 tests/xfs/1902.out create mode 100755 tests/xfs/1903 create mode 100644 tests/xfs/1903.out
PATCH 1/14 xfs: test health monitoring code
From: Darrick J. Wong <[email protected]>
Add some functionality tests for the new health monitoring code.
--- common/module | 11 ++++++++++ tests/xfs/1885 | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1885.out | 5 ++++ 3 files changed, 75 insertions(+) create mode 100755 tests/xfs/1885 create mode 100644 tests/xfs/1885.out
common/module +11 -0
@@ -225,3 +225,14 @@ _optional_reload_fs_module() _test_loadable_fs_module "$@" 2>&1 | \ sed -e '/patient module removal/d' } + +_require_module_refcount() +{ + local refcount_file="/sys/module/$1/refcnt" + test -e "$refcount_file" || _notrun "cannot find $1 module refcount" +} + +_module_refcount() +{ + cat "/sys/module/$1/refcnt" +}
tests/xfs/1885 new file +59 -0
59 lines not on this page · raw message
tests/xfs/1885.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 1885 +mount refcount is in range +mount + healthmon refcount is in range +healthmon refcount is in range +end refcount is in range
Re: [PATCH 01/14] xfs: test health monitoring code
Quoted 46 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add some functionality tests for the new health monitoring code.
Signed-off-by "Darrick J. Wong" <[email protected]>--- common/module | 11 ++++++++++ tests/xfs/1885 | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1885.out | 5 ++++ 3 files changed, 75 insertions(+) create mode 100755 tests/xfs/1885 create mode 100644 tests/xfs/1885.outcommon/module +11
-0@@ -225,3 +225,14 @@ _optional_reload_fs_module() _test_loadable_fs_module "$@" 2>&1 | \ sed -e '/patient module removal/d' } + +_require_module_refcount() +{ + local refcount_file="/sys/module/$1/refcnt" + test -e "$refcount_file" || _notrun "cannot find $1 module refcount" +} + +_module_refcount() +{ + cat "/sys/module/$1/refcnt" +}tests/xfs/1885 new file +7
-0@@ -0,0 +1,59 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1885 +# +# Make sure that healthmon handles module refcount correctly.
tests/xfs/1885 line 17
Quoted 10 lines
tests/xfs/1885 new file +10
-0+# +. ./common/preamble +_begin_fstest auto selfhealing quick + +. ./common/filter +. ./common/module + +_cleanup() +{ + test -n "$healer_pid" && kill $healer_pid &>/dev/null
I'll add a "wait" at here, and ...
Quoted 23 lines
tests/xfs/1885 new file +23
-0+ cd / + rm -r -f $tmp.* +} + +_require_test +_require_xfs_io_command healthmon +_require_module_refcount xfs + +# Capture mod refcount without the test fs mounted +_test_unmount +init_refcount="$(_module_refcount xfs)" + +# Capture mod refcount with the test fs mounted +_test_mount +nomon_mount_refcount="$(_module_refcount xfs)" + +# Capture mod refcount with test fs mounted and the healthmon fd open. +# Pause the xfs_io process so that it doesn't actually respond to events. +$XFS_IO_PROG -c 'healthmon -c -v' $TEST_DIR >> $seqres.full & +healer_pid=$! +sleep 0.5 +kill -STOP $healer_pid +mon_mount_refcount="$(_module_refcount xfs)"
tests/xfs/1885 line 50
Quoted 10 lines
tests/xfs/1885 new file +10
-0+ +# Capture mod refcount with only the healthmon fd open. +_test_unmount +mon_nomount_refcount="$(_module_refcount xfs)" + +# Capture mod refcount after continuing healthmon (which should exit due to the +# unmount) and killing it. +kill -CONT $healer_pid +kill $healer_pid +wait
unset healer_pid
others look good to me,
Quoted 21 lines
tests/xfs/1885 new file +9
-0+nomon_nomount_refcount="$(_module_refcount xfs)" + +_within_tolerance "mount refcount" "$nomon_mount_refcount" "$((init_refcount + 1))" 0 -v +_within_tolerance "mount + healthmon refcount" "$mon_mount_refcount" "$((init_refcount + 2))" 0 -v +_within_tolerance "healthmon refcount" "$mon_nomount_refcount" "$((init_refcount + 1))" 0 -v +_within_tolerance "end refcount" "$nomon_nomount_refcount" "$init_refcount" 0 -v + +status=0 +exittests/xfs/1885.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 1885 +mount refcount is in range +mount + healthmon refcount is in range +healthmon refcount is in range +end refcount is in range
PATCH 2/14 xfs: test for metadata corruption error reporting via healthmon
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check if we can detect runtime metadata corruptions via the health monitor.
--- tests/xfs/1879 | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1879.out | 8 ++++ 2 files changed, 101 insertions(+) create mode 100755 tests/xfs/1879 create mode 100644 tests/xfs/1879.out
tests/xfs/1879 new file +93 -0
@@ -0,0 +1,93 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1879 +# +# Corrupt some metadata and try to access it with the health monitoring program +# running. Check that healthmon observes a metadata error. +# +. ./common/preamble +_begin_fstest auto quick eio selfhealing + +_cleanup() +{ + cd / + rm -rf $tmp.* $testdir +} + +. ./common/filter + +_require_scratch_nocheck +_require_scratch_xfs_crc # can't detect minor corruption w/o crc +_require_xfs_io_command healthmon + +# Disable the scratch rt device to avoid test failures relating to the rt +# bitmap consuming all the free space in our small data device. +unset SCRATCH_RTDEV + +echo "Format and mount" +_scratch_mkfs -d agcount=1 | _filter_mkfs 2> $tmp.mkfs >> $seqres.full +. $tmp.mkfs +_scratch_mount +mkdir $SCRATCH_MNT/a/ +# Enough entries to get to a single block directory +for ((i = 0; i < ( (isize + 255) / 256); i++)); do + path="$(printf "%s/a/%0255d" "$SCRATCH_MNT" "$i")" + touch "$path" +done +inum="$(stat -c %i "$SCRATCH_MNT/a")" +_scratch_unmount + +# Fuzz the directory block so that the touch below will be guaranteed to trip +# a runtime sickness report in exactly the manner we desire. +_scratch_xfs_db -x -c "inode $inum" -c "dblock 0" -c 'fuzz bhdr.hdr.owner add' -c print &>> $seqres.full + +# Try to allocate space to trigger a metadata corruption event +echo "Runtime corruption detection" +_scratch_mount +$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT > $tmp.healthmon & +sleep 1 # wait for program to start up +touch $SCRATCH_MNT/a/farts &>> $seqres.full +_scratch_unmount + +wait # for healthmon to finish + +# Did we get errors? +check_healthmon() +{ + cat $tmp.healthmon >> $seqres.full + _filter_scratch < $tmp.healthmon | \ + grep -E '(sick|corrupt)' | \ + sed -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino [0-9]* gen 0x[0-9a-f]*|VICTIM|g' | \ + sort | \ + uniq +} +check_healthmon + +# Run scrub to trigger a health event from there too. +echo "Scrub corruption detection" +_scratch_mount +if _supports_xfs_scrub $SCRATCH_MNT $SCRATCH_DEV; then + $XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT > $tmp.healthmon & + sleep 1 # wait for program to start up + $XFS_SCRUB_PROG -n $SCRATCH_MNT &>> $seqres.full + _scratch_unmount + + wait # for healthmon to finish + + # Did we get errors? + check_healthmon +else + # mock the output since we don't support scrub + _scratch_unmount + cat << ENDL +VICTIM directory: corrupt +VICTIM directory: sick +VICTIM parent: corrupt +ENDL +fi + +status=0 +exit
tests/xfs/1879.out new file +8 -0
@@ -0,0 +1,8 @@ +QA output created by 1879 +Format and mount +Runtime corruption detection +VICTIM directory: sick +Scrub corruption detection +VICTIM directory: corrupt +VICTIM directory: sick +VICTIM parent: corrupt
Re: [PATCH 02/14] xfs: test for metadata corruption error reporting via healthmon
Quoted 25 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Check if we can detect runtime metadata corruptions via the health monitor.
Signed-off-by "Darrick J. Wong" <[email protected]>--- tests/xfs/1879 | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1879.out | 8 ++++ 2 files changed, 101 insertions(+) create mode 100755 tests/xfs/1879 create mode 100644 tests/xfs/1879.outtests/xfs/1879 new file +5
-0@@ -0,0 +1,93 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1879
tests/xfs/1879 line 15
Quoted 10 lines
tests/xfs/1879 new file +10
-0+# +# Corrupt some metadata and try to access it with the health monitoring program +# running. Check that healthmon observes a metadata error. +# +. ./common/preamble +_begin_fstest auto quick eio selfhealing + +_cleanup() +{ + cd /
test -n "$healer_pid" && kill $healer_pid &>/dev/null wait
Quoted 24 lines
tests/xfs/1879 new file +24
-0+ rm -rf $tmp.* $testdir +} + +. ./common/filter + +_require_scratch_nocheck +_require_scratch_xfs_crc # can't detect minor corruption w/o crc +_require_xfs_io_command healthmon + +# Disable the scratch rt device to avoid test failures relating to the rt +# bitmap consuming all the free space in our small data device. +unset SCRATCH_RTDEV + +echo "Format and mount" +_scratch_mkfs -d agcount=1 | _filter_mkfs 2> $tmp.mkfs >> $seqres.full +. $tmp.mkfs +_scratch_mount +mkdir $SCRATCH_MNT/a/ +# Enough entries to get to a single block directory +for ((i = 0; i < ( (isize + 255) / 256); i++)); do + path="$(printf "%s/a/%0255d" "$SCRATCH_MNT" "$i")" + touch "$path" +done +inum="$(stat -c %i "$SCRATCH_MNT/a")"
tests/xfs/1879 line 49
Quoted 10 lines
tests/xfs/1879 new file +10
-0+_scratch_unmount + +# Fuzz the directory block so that the touch below will be guaranteed to trip +# a runtime sickness report in exactly the manner we desire. +_scratch_xfs_db -x -c "inode $inum" -c "dblock 0" -c 'fuzz bhdr.hdr.owner add' -c print &>> $seqres.full + +# Try to allocate space to trigger a metadata corruption event +echo "Runtime corruption detection" +_scratch_mount +$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT > $tmp.healthmon &
healer_pid=$!
tests/xfs/1879 line 54
Quoted 5 lines
tests/xfs/1879 new file +5
-0+sleep 1 # wait for program to start up +touch $SCRATCH_MNT/a/farts &>> $seqres.full +_scratch_unmount + +wait # for healthmon to finish
unset healer_pid
Others look good to me,
Quoted 54 lines
tests/xfs/1879 new file +39
-0+ +# Did we get errors? +check_healthmon() +{ + cat $tmp.healthmon >> $seqres.full + _filter_scratch < $tmp.healthmon | \ + grep -E '(sick|corrupt)' | \ + sed -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino [0-9]* gen 0x[0-9a-f]*|VICTIM|g' | \ + sort | \ + uniq +} +check_healthmon + +# Run scrub to trigger a health event from there too. +echo "Scrub corruption detection" +_scratch_mount +if _supports_xfs_scrub $SCRATCH_MNT $SCRATCH_DEV; then + $XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT > $tmp.healthmon & + sleep 1 # wait for program to start up + $XFS_SCRUB_PROG -n $SCRATCH_MNT &>> $seqres.full + _scratch_unmount + + wait # for healthmon to finish + + # Did we get errors? + check_healthmon +else + # mock the output since we don't support scrub + _scratch_unmount + cat << ENDL +VICTIM directory: corrupt +VICTIM directory: sick +VICTIM parent: corrupt +ENDL +fi + +status=0 +exittests/xfs/1879.out new file +8
-0@@ -0,0 +1,8 @@ +QA output created by 1879 +Format and mount +Runtime corruption detection +VICTIM directory: sick +Scrub corruption detection +VICTIM directory: corrupt +VICTIM directory: sick +VICTIM parent: corrupt
PATCH 3/14 xfs: test io error reporting via healthmon
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new test to make sure the kernel can report IO errors via health monitoring.
--- tests/xfs/1878 | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1878.out | 10 ++++++ 2 files changed, 103 insertions(+) create mode 100755 tests/xfs/1878 create mode 100644 tests/xfs/1878.out
tests/xfs/1878 new file +93 -0
@@ -0,0 +1,93 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1878 +# +# Attempt to read and write a file in buffered and directio mode with the +# health monitoring program running. Check that healthmon observes all four +# types of IO errors. +# +. ./common/preamble +_begin_fstest auto quick eio selfhealing + +_cleanup() +{ + cd / + rm -rf $tmp.* $testdir + _dmerror_cleanup +} + +. ./common/filter +. ./common/dmerror + +_require_scratch_nocheck +_require_xfs_io_command healthmon +_require_dm_target error + +filter_healer_errors() { + _filter_scratch | \ + grep -E '(buffered|directio)' | \ + sed \ + -e 's/ino [0-9]*/ino NUM/g' \ + -e 's/gen 0x[0-9a-f]*/gen NUM/g' \ + -e 's/pos [0-9]*/pos NUM/g' \ + -e 's/len [0-9]*/len NUM/g' \ + -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' | \ + uniq +} + +# Disable the scratch rt device to avoid test failures relating to the rt +# bitmap consuming all the free space in our small data device. +unset SCRATCH_RTDEV + +echo "Format and mount" +_scratch_mkfs > $seqres.full 2>&1 +_dmerror_init no_log +_dmerror_mount + +_require_fs_space $SCRATCH_MNT 65536 + +# Create a file with written regions far enough apart that the pagecache can't +# possibly be caching the regions with a single folio. +testfile=$SCRATCH_MNT/fsync-err-test +$XFS_IO_PROG -f \ + -c 'pwrite -b 1m 0 1m' \ + -c 'pwrite -b 1m 10g 1m' \ + -c 'pwrite -b 1m 20g 1m' \ + -c fsync $testfile >> $seqres.full + +# First we check if directio errors get reported +$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT >> $tmp.healthmon & +sleep 1 # wait for program to start up +_dmerror_load_error_table +$XFS_IO_PROG -d -c 'pwrite -b 256k 12k 16k' $testfile >> $seqres.full +$XFS_IO_PROG -d -c 'pread -b 256k 10g 16k' $testfile >> $seqres.full +_dmerror_load_working_table + +_dmerror_unmount +wait # for healthmon to finish +_dmerror_mount + +# Next we check if buffered io errors get reported. We have to write something +# before loading the error table to ensure the dquots get loaded. +$XFS_IO_PROG -c 'pwrite -b 256k 20g 1k' -c fsync $testfile >> $seqres.full +$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT >> $tmp.healthmon & +sleep 1 # wait for program to start up +_dmerror_load_error_table +$XFS_IO_PROG -c 'pread -b 256k 12k 16k' $testfile >> $seqres.full +$XFS_IO_PROG -c 'pwrite -b 256k 20g 16k' -c fsync $testfile >> $seqres.full +_dmerror_load_working_table + +_dmerror_unmount +wait # for healthmon to finish + +# Did we get errors? +cat $tmp.healthmon >> $seqres.full +filter_healer_errors < $tmp.healthmon + +_dmerror_cleanup + +status=0 +exit
tests/xfs/1878.out new file +10 -0
@@ -0,0 +1,10 @@ +QA output created by 1878 +Format and mount +pwrite: Input/output error +pread: Input/output error +pread: Input/output error +fsync: Input/output error +VICTIM pos NUM len NUM: directio_write: Input/output error +VICTIM pos NUM len NUM: directio_read: Input/output error +VICTIM pos NUM len NUM: buffered_read: Input/output error +VICTIM pos NUM len NUM: buffered_write: Input/output error
Re: [PATCH 03/14] xfs: test io error reporting via healthmon
Quoted 26 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Create a new test to make sure the kernel can report IO errors via health monitoring.
Signed-off-by "Darrick J. Wong" <[email protected]>--- tests/xfs/1878 | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1878.out | 10 ++++++ 2 files changed, 103 insertions(+) create mode 100755 tests/xfs/1878 create mode 100644 tests/xfs/1878.outtests/xfs/1878 new file +6
-0@@ -0,0 +1,93 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1878 +#
tests/xfs/1878 line 16
Quoted 10 lines
tests/xfs/1878 new file +10
-0+# Attempt to read and write a file in buffered and directio mode with the +# health monitoring program running. Check that healthmon observes all four +# types of IO errors. +# +. ./common/preamble +_begin_fstest auto quick eio selfhealing + +_cleanup() +{ + cd /
test -n "$healer_pid" && kill $healer_pid &>/dev/null wait
Quoted 36 lines
tests/xfs/1878 new file +36
-0+ rm -rf $tmp.* $testdir + _dmerror_cleanup +} + +. ./common/filter +. ./common/dmerror + +_require_scratch_nocheck +_require_xfs_io_command healthmon +_require_dm_target error + +filter_healer_errors() { + _filter_scratch | \ + grep -E '(buffered|directio)' | \ + sed \ + -e 's/ino [0-9]*/ino NUM/g' \ + -e 's/gen 0x[0-9a-f]*/gen NUM/g' \ + -e 's/pos [0-9]*/pos NUM/g' \ + -e 's/len [0-9]*/len NUM/g' \ + -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' | \ + uniq +} + +# Disable the scratch rt device to avoid test failures relating to the rt +# bitmap consuming all the free space in our small data device. +unset SCRATCH_RTDEV + +echo "Format and mount" +_scratch_mkfs > $seqres.full 2>&1 +_dmerror_init no_log +_dmerror_mount + +_require_fs_space $SCRATCH_MNT 65536 + +# Create a file with written regions far enough apart that the pagecache can't
tests/xfs/1878 line 62
Quoted 10 lines
tests/xfs/1878 new file +10
-0+# possibly be caching the regions with a single folio. +testfile=$SCRATCH_MNT/fsync-err-test +$XFS_IO_PROG -f \ + -c 'pwrite -b 1m 0 1m' \ + -c 'pwrite -b 1m 10g 1m' \ + -c 'pwrite -b 1m 20g 1m' \ + -c fsync $testfile >> $seqres.full + +# First we check if directio errors get reported +$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT >> $tmp.healthmon &
healer_pid=$!
tests/xfs/1878 line 70
Quoted 8 lines
tests/xfs/1878 new file +8
-0+sleep 1 # wait for program to start up +_dmerror_load_error_table +$XFS_IO_PROG -d -c 'pwrite -b 256k 12k 16k' $testfile >> $seqres.full +$XFS_IO_PROG -d -c 'pread -b 256k 10g 16k' $testfile >> $seqres.full +_dmerror_load_working_table + +_dmerror_unmount +wait # for healthmon to finish
unset healer_pid
tests/xfs/1878 line 76
Quoted 6 lines
tests/xfs/1878 new file +6
-0+_dmerror_mount + +# Next we check if buffered io errors get reported. We have to write something +# before loading the error table to ensure the dquots get loaded. +$XFS_IO_PROG -c 'pwrite -b 256k 20g 1k' -c fsync $testfile >> $seqres.full +$XFS_IO_PROG -c 'healthmon -c -v' $SCRATCH_MNT >> $tmp.healthmon &
healer_pid=$!
tests/xfs/1878 line 84
Quoted 8 lines
tests/xfs/1878 new file +8
-0+sleep 1 # wait for program to start up +_dmerror_load_error_table +$XFS_IO_PROG -c 'pread -b 256k 12k 16k' $testfile >> $seqres.full +$XFS_IO_PROG -c 'pwrite -b 256k 20g 16k' -c fsync $testfile >> $seqres.full +_dmerror_load_working_table + +_dmerror_unmount +wait # for healthmon to finish
unset healer_pid
tests/xfs/1878 line 93
Quoted 9 lines
tests/xfs/1878 new file +9
-0+ +# Did we get errors? +cat $tmp.healthmon >> $seqres.full +filter_healer_errors < $tmp.healthmon + +_dmerror_cleanup + +status=0 +exit
_exit 0
Others look good to me, I'll change these when I merge this patch.
Quoted 17 lines
tests/xfs/1878.out new file +10
-0@@ -0,0 +1,10 @@ +QA output created by 1878 +Format and mount +pwrite: Input/output error +pread: Input/output error +pread: Input/output error +fsync: Input/output error +VICTIM pos NUM len NUM: directio_write: Input/output error +VICTIM pos NUM len NUM: directio_read: Input/output error +VICTIM pos NUM len NUM: buffered_read: Input/output error +VICTIM pos NUM len NUM: buffered_write: Input/output error
PATCH 4/14 xfs: set up common code for testing xfs_healer
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a bunch of common code so that we can test the xfs_healer daemon. Most of the changes here are to make it easier to manage the systemd service units for xfs_healer and xfs_scrub.
--- common/config | 14 ++++++++ common/rc | 5 +++ common/systemd | 39 ++++++++++++++++++++++ common/xfs | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/802 | 4 +- 5 files changed, 158 insertions(+), 2 deletions(-)
common/config +14 -0
@@ -161,6 +161,20 @@ export XFS_ADMIN_PROG="$(type -P xfs_admin)" export XFS_GROWFS_PROG=$(type -P xfs_growfs) export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)" export XFS_SCRUB_PROG="$(type -P xfs_scrub)" + +XFS_HEALER_PROG="$(type -P xfs_healer)" +XFS_HEALER_START_PROG="$(type -P xfs_healer_start)" + +# If not found, try the ones installed in libexec +if [ ! -x "$XFS_HEALER_PROG" ] && [ -e /usr/libexec/xfsprogs/xfs_healer ]; then + XFS_HEALER_PROG=/usr/libexec/xfsprogs/xfs_healer +fi +if [ ! -x "$XFS_HEALER_START_PROG" ] && [ -e /usr/libexec/xfsprogs/xfs_healer_start ]; then + XFS_HEALER_START_PROG=/usr/libexec/xfsprogs/xfs_healer_start +fi +export XFS_HEALER_PROG +export XFS_HEALER_START_PROG + export XFS_PARALLEL_REPAIR_PROG="$(type -P xfs_prepair)" export XFS_PARALLEL_REPAIR64_PROG="$(type -P xfs_prepair64)" export __XFSDUMP_PROG="$(type -P xfsdump)"
common/rc +5 -0
@@ -3021,6 +3021,11 @@ _require_xfs_io_command() "label") testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1` ;; + "verifymedia") + testio=`$XFS_IO_PROG -x -c "verifymedia $* 0 0" 2>&1` + echo $testio | grep -q "invalid option" && \ + _notrun "xfs_io $command support is missing" + ;; "open") # -c "open $f" is broken in xfs_io <= 4.8. Along with the fix, # a new -C flag was introduced to execute one shot commands.
common/systemd +39 -0
@@ -44,6 +44,18 @@ _systemd_unit_active() { test "$(systemctl is-active "$1")" = "active" } +# Wait for up to a certain number of seconds for a service to reach inactive +# state. +_systemd_unit_wait() { + local svcname="$1" + local timeout="${2:-30}" + + for ((i = 0; i < (timeout * 2); i++)); do + test "$(systemctl is-active "$svcname")" = "inactive" && break + sleep 0.5 + done +} + _require_systemd_unit_active() { _require_systemd_unit_defined "$1" _systemd_unit_active "$1" || \
@@ -71,3 +83,30 @@ _systemd_unit_status() { _systemd_installed || return 1 systemctl status "$1" } + +# Start a running systemd unit +_systemd_unit_start() { + systemctl start "$1" +} +# Stop a running systemd unit +_systemd_unit_stop() { + systemctl stop "$1" +} + +# Mask or unmask a running systemd unit +_systemd_unit_mask() { + systemctl mask "$1" +} +_systemd_unit_unmask() { + systemctl unmask "$1" +} +_systemd_unit_masked() { + systemctl status "$1" 2>/dev/null | grep -q 'Loaded: masked' +} + +_systemd_service_unit_path() { + local template="$1" + local path="$2" + + systemd-escape --template "$template" --path "$path" +}
common/xfs +98 -0
101 lines not on this page · raw message
tests/xfs/802 +2 -2
@@ -105,8 +105,8 @@ run_scrub_service() { } echo "Scrub Scratch FS" -scratch_path=$(systemd-escape --path "$SCRATCH_MNT") -run_scrub_service xfs_scrub@$scratch_path +svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")" +run_scrub_service "$svc" find_scrub_trace "$SCRATCH_MNT" # Remove the xfs_scrub_all media scan stamp directory (if specified) because we
Re: [PATCH 04/14] xfs: set up common code for testing xfs_healer
Quoted 8 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add a bunch of common code so that we can test the
xfs_healerdaemon. Most of the changes here are to make it easier to manage the systemd service units forxfs_healerandxfs_scrub.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 225 lines
common/config | 14 ++++++++ common/rc | 5 +++ common/systemd | 39 ++++++++++++++++++++++ common/xfs | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/802 | 4 +- 5 files changed, 158 insertions(+), 2 deletions(-)common/config +14
-0@@ -161,6 +161,20 @@ export XFS_ADMIN_PROG="$(type -P xfs_admin)" export XFS_GROWFS_PROG=$(type -P xfs_growfs) export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)" export XFS_SCRUB_PROG="$(type -P xfs_scrub)" + +XFS_HEALER_PROG="$(type -P xfs_healer)" +XFS_HEALER_START_PROG="$(type -P xfs_healer_start)" + +# If not found, try the ones installed in libexec +if [ ! -x "$XFS_HEALER_PROG" ] && [ -e /usr/libexec/xfsprogs/xfs_healer ]; then + XFS_HEALER_PROG=/usr/libexec/xfsprogs/xfs_healer +fi +if [ ! -x "$XFS_HEALER_START_PROG" ] && [ -e /usr/libexec/xfsprogs/xfs_healer_start ]; then + XFS_HEALER_START_PROG=/usr/libexec/xfsprogs/xfs_healer_start +fi +export XFS_HEALER_PROG +export XFS_HEALER_START_PROG + export XFS_PARALLEL_REPAIR_PROG="$(type -P xfs_prepair)" export XFS_PARALLEL_REPAIR64_PROG="$(type -P xfs_prepair64)" export __XFSDUMP_PROG="$(type -P xfsdump)"common/rc +5
-0@@ -3021,6 +3021,11 @@ _require_xfs_io_command() "label") testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1` ;; + "verifymedia") + testio=`$XFS_IO_PROG -x -c "verifymedia $* 0 0" 2>&1` + echo $testio | grep -q "invalid option" && \ + _notrun "xfs_io $command support is missing" + ;; "open") # -c "open $f" is broken in xfs_io <= 4.8. Along with the fix, # a new -C flag was introduced to execute one shot commands.common/systemd +39
-0@@ -44,6 +44,18 @@ _systemd_unit_active() { test "$(systemctl is-active "$1")" = "active" } +# Wait for up to a certain number of seconds for a service to reach inactive +# state. +_systemd_unit_wait() { + local svcname="$1" + local timeout="${2:-30}" + + for ((i = 0; i < (timeout * 2); i++)); do + test "$(systemctl is-active "$svcname")" = "inactive" && break + sleep 0.5 + done +} + _require_systemd_unit_active() { _require_systemd_unit_defined "$1" _systemd_unit_active "$1" || \@@ -71,3 +83,30 @@ _systemd_unit_status() { _systemd_installed || return 1 systemctl status "$1" } + +# Start a running systemd unit +_systemd_unit_start() { + systemctl start "$1" +} +# Stop a running systemd unit +_systemd_unit_stop() { + systemctl stop "$1" +} + +# Mask or unmask a running systemd unit +_systemd_unit_mask() { + systemctl mask "$1" +} +_systemd_unit_unmask() { + systemctl unmask "$1" +} +_systemd_unit_masked() { + systemctl status "$1" 2>/dev/null | grep -q 'Loaded: masked' +} + +_systemd_service_unit_path() { + local template="$1" + local path="$2" + + systemd-escape --template "$template" --path "$path" +}common/xfs +98
-0@@ -2301,3 +2301,101 @@ _filter_bmap_gno() if ($ag =~ /\d+/) {print "$ag "} ; ' } + +# Run the xfs_healer program on some filesystem +_xfs_healer() { + $XFS_HEALER_PROG "$@" +} + +# Compute the xfs_healer systemd service instance name for a given path. +# This is easy because xfs_healer has always supported --svcname. +_xfs_healer_svcname() +{ + _xfs_healer --svcname "$@" +} + +# Compute the xfs_scrub systemd service instance name for a given path. This +# is tricky because xfs_scrub only gained --svcname when xfs_healer was made. +_xfs_scrub_svcname() +{ + local ret + + if ret="$($XFS_SCRUB_PROG --svcname "$@")"; then + echo "$ret" + return 0 + fi + + # ...but if not, we can fall back to brute force systemd invocations. + _systemd_service_unit_path "[email protected]" "$*" +} + +# Run the xfs_healer program on the scratch fs +_scratch_xfs_healer() { + _xfs_healer "$@" "$SCRATCH_MNT" +} + +# Turn off the background xfs_healer service if any so that it doesn't fix +# injected metadata errors; then start a background copy of xfs_healer to +# capture that. +_invoke_xfs_healer() { + local mount="$1" + local logfile="$2" + shift; shift + + if _systemd_is_running; then + local svc="$(_xfs_healer_svcname "$mount")" + _systemd_unit_stop "$svc" &>> $seqres.full + fi + + $XFS_HEALER_PROG "$mount" "$@" &> "$logfile" & + XFS_HEALER_PID=$! + + # Wait 30s for the healer program to really start up + for ((i = 0; i < 60; i++)); do + test -e "$logfile" && \ + grep -q 'monitoring started' "$logfile" && \ + break + sleep 0.5 + done +} + +# Run our own copy of xfs_healer against the scratch device. Note that +# unmounting the scratch fs causes the healer daemon to exit, so we don't need +# to kill it explicitly from _cleanup. +_scratch_invoke_xfs_healer() { + _invoke_xfs_healer "$SCRATCH_MNT" "$@" +} + +# Unmount the filesystem to kill the xfs_healer instance started by +# _invoke_xfs_healer, and wait up to a certain amount of time for it to exit. +_kill_xfs_healer() { + local unmount="$1" + local timeout="${2:-30}" + local i + + # Unmount fs to kill healer, then wait for it to finish + for ((i = 0; i < (timeout * 2); i++)); do + $unmount &>> $seqres.full && break + sleep 0.5 + done + + test -n "$XFS_HEALER_PID" && \ + kill $XFS_HEALER_PID &>> $seqres.full + wait + unset XFS_HEALER_PID +} + +# Unmount the scratch fs to kill a _scratch_invoke_xfs_healer instance. +_scratch_kill_xfs_healer() { + local unmount="${1:-_scratch_unmount}" + shift + + _kill_xfs_healer "$unmount" "$@" +} + +# Does this mounted filesystem support xfs_healer? +_require_xfs_healer() +{ + _xfs_healer --supported "$@" &>/dev/null || \ + _notrun "health monitoring not supported on this kernel" +}tests/xfs/802 +2
-2@@ -105,8 +105,8 @@ run_scrub_service() { } echo "Scrub Scratch FS" -scratch_path=$(systemd-escape --path "$SCRATCH_MNT") -run_scrub_service xfs_scrub@$scratch_path +svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")" +run_scrub_service "$svc" find_scrub_trace "$SCRATCH_MNT" # Remove the xfs_scrub_all media scan stamp directory (if specified) because we
Re: [PATCH 04/14] xfs: set up common code for testing xfs_healer
In reply to Darrick J. WongQuoted 15 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add a bunch of common code so that we can test the
xfs_healerdaemon. Most of the changes here are to make it easier to manage the systemd service units forxfs_healerandxfs_scrub.Signed-off-by "Darrick J. Wong" <[email protected]>--- common/config | 14 ++++++++ common/rc | 5 +++ common/systemd | 39 ++++++++++++++++++++++ common/xfs | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/802 | 4 +- 5 files changed, 158 insertions(+), 2 deletions(-)
[snip]
common/xfs _filter_bmap_gno() · line 2323
Quoted 7 lines
common/xfs +7
-0+# Compute the xfs_scrub systemd service instance name for a given path. This +# is tricky because xfs_scrub only gained --svcname when xfs_healer was made. +_xfs_scrub_svcname() +{ + local ret + + if ret="$($XFS_SCRUB_PROG --svcname "$@")"; then
Better to be:
- if ret="$($XFS_SCRUB_PROG --svcname "$@")"; then
+ if ret="$($XFS_SCRUB_PROG --svcname "$@" 2>/dev/null)"; then
Or below xfs/802 will ...
Quoted 3 lines
common/xfs +3
-0+ echo "$ret" + return 0 + fi
[snip]
tests/xfs/802 run_scrub_service() { · line 108
Quoted 11 lines
tests/xfs/802 +1
-2@@ -105,8 +105,8 @@ run_scrub_service() { } echo "Scrub Scratch FS" -scratch_path=$(systemd-escape --path "$SCRATCH_MNT") -run_scrub_service xfs_scrub@$scratch_path +svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")"
... fails on old xfsprogs as:
--- /dev/fd/63 2026-03-13 19:16:15.217899866 -0400
+++ xfs/802.out.bad 2026-03-13 19:16:15.191834546 -0400
@@ -1,5 +1,21 @@
QA output created by 802
Format and populate
Scrub Scratch FS
+/usr/sbin/xfs_scrub: invalid option -- '-'
+Usage: xfs_scrub [OPTIONS] mountpoint
+
+Options:
+ -a count Stop after this many errors are found. + -b Background mode. + -C fd Print progress information to this fd. + -e behavior What to do if errors are found. + -k Do not FITRIM the free space. + -m path Path to /etc/mtab. + -n Dry run. Do not modify anything. + -p Only optimize, do not fix corruptions. + -T Display timing/usage information. + -v Verbose output. + -V Print version. + -x Scrub file data too.
Scrub Everything
Scrub Done
If you don't have more suggestion, I'll help to change that :)
Thanks,
Zorro
Quoted 5 lines
tests/xfs/802 +1
-0+run_scrub_service "$svc" find_scrub_trace "$SCRATCH_MNT" # Remove the xfs_scrub_all media scan stamp directory (if specified) because we
Re: [PATCH 04/14] xfs: set up common code for testing xfs_healer
Quoted 76 lines · Zorro Lang
Quoted 15 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add a bunch of common code so that we can test the
xfs_healerdaemon. Most of the changes here are to make it easier to manage the systemd service units forxfs_healerandxfs_scrub.Signed-off-by "Darrick J. Wong" <[email protected]>--- common/config | 14 ++++++++ common/rc | 5 +++ common/systemd | 39 ++++++++++++++++++++++ common/xfs | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/802 | 4 +- 5 files changed, 158 insertions(+), 2 deletions(-)[snip]
common/xfs _filter_bmap_gno() · line 2323
Quoted 7 lines
common/xfs +7
-0+# Compute the xfs_scrub systemd service instance name for a given path. This +# is tricky because xfs_scrub only gained --svcname when xfs_healer was made. +_xfs_scrub_svcname() +{ + local ret + + if ret="$($XFS_SCRUB_PROG --svcname "$@")"; thenBetter to be:
(quoted diff) +1
-1- if ret="$($XFS_SCRUB_PROG --svcname "$@")"; then + if ret="$($XFS_SCRUB_PROG --svcname "$@" 2>/dev/null)"; thenOr below xfs/802 will ...
Quoted 3 lines
common/xfs +3
-0+ echo "$ret" + return 0 + fi[snip]
tests/xfs/802 run_scrub_service() { · line 108
Quoted 11 lines
tests/xfs/802 +1
-2@@ -105,8 +105,8 @@ run_scrub_service() { } echo "Scrub Scratch FS" -scratch_path=$(systemd-escape --path "$SCRATCH_MNT") -run_scrub_service xfs_scrub@$scratch_path +svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")"... fails on old xfsprogs as:
---
/dev/fd/632026-03-13 19:16:15.217899866 -0400
+++ xfs/802.out.bad 2026-03-13 19:16:15.191834546 -0400
@@ -1,5 +1,21 @@
QA output created by 802
Format and populate
Scrub Scratch FS
+/usr/sbin/xfs_scrub: invalid option -- '-'
+Usage:xfs_scrub[OPTIONS] mountpoint
+
+Options:+ -a count Stop after this many errors are found. + -b Background mode. + -C fd Print progress information to this fd. + -e behavior What to do if errors are found. + -k Do not FITRIM the free space. + -m path Path to /etc/mtab. + -n Dry run. Do not modify anything. + -p Only optimize, do not fix corruptions. + -T Display timing/usage information. + -v Verbose output. + -V Print version. + -x Scrub file data too.
Quoted 4 lines
Scrub Everything
Scrub DoneIf you don't have more suggestion, I'll help to change that :)
That seems the proper correction to make. Thanks for your help!
--D
Quoted 9 lines
Thanks,
ZorroQuoted 5 lines
tests/xfs/802 +1
-0+run_scrub_service "$svc" find_scrub_trace "$SCRATCH_MNT" # Remove the xfs_scrub_all media scan stamp directory (if specified) because we
PATCH 5/14 xfs: test xfs_healer's event handling
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that xfs_healer can handle every type of event that the kernel can throw at it by initiating a full scrub of a test filesystem.
--- tests/xfs/1882 | 44 ++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1882.out | 2 ++ 2 files changed, 46 insertions(+) create mode 100755 tests/xfs/1882 create mode 100644 tests/xfs/1882.out
tests/xfs/1882 new file +44 -0
@@ -0,0 +1,44 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1882 +# +# Make sure that xfs_healer correctly handles all the reports that it gets +# from the kernel. We simulate this by using the --everything mode so we get +# all the events, not just the sickness reports. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd +. ./common/populate + +_require_scrub +_require_xfs_io_command "scrub" # online check support +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_scratch + +# Does this fs support health monitoring? +_scratch_mkfs >> $seqres.full +_scratch_mount +_require_xfs_healer $SCRATCH_MNT +_scratch_unmount + +# Create a sample fs with all the goodies +_scratch_populate_cached nofill &>> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --everything + +# Run scrub to make some noise +_scratch_scrub -b -n >> $seqres.full + +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full + +echo Silence is golden +status=0 +exit
tests/xfs/1882.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 1882 +Silence is golden
Re: [PATCH 05/14] xfs: test xfs_healer's event handling
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that
xfs_healercan handle every type of event that the kernel can throw at it by initiating a full scrub of a test filesystem.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 66 lines
tests/xfs/1882 | 44 ++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1882.out | 2 ++ 2 files changed, 46 insertions(+) create mode 100755 tests/xfs/1882 create mode 100644 tests/xfs/1882.outtests/xfs/1882 new file +44
-0@@ -0,0 +1,44 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1882 +# +# Make sure that xfs_healer correctly handles all the reports that it gets +# from the kernel. We simulate this by using the --everything mode so we get +# all the events, not just the sickness reports. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd +. ./common/populate + +_require_scrub +_require_xfs_io_command "scrub" # online check support +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_scratch + +# Does this fs support health monitoring? +_scratch_mkfs >> $seqres.full +_scratch_mount +_require_xfs_healer $SCRATCH_MNT +_scratch_unmount + +# Create a sample fs with all the goodies +_scratch_populate_cached nofill &>> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --everything + +# Run scrub to make some noise +_scratch_scrub -b -n >> $seqres.full + +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full + +echo Silence is golden +status=0 +exittests/xfs/1882.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 1882 +Silence is golden
PATCH 6/14 xfs: test xfs_healer can fix a filesystem
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that xfs_healer can actually fix an injected metadata corruption.
--- tests/xfs/1884 | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1884.out | 2 + 2 files changed, 91 insertions(+) create mode 100755 tests/xfs/1884 create mode 100644 tests/xfs/1884.out
tests/xfs/1884 new file +89 -0
@@ -0,0 +1,89 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1884 +# +# Ensure that autonomous self healing fixes the filesystem correctly. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --repair + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err + +# List the dirents of /victimdir to see if it stops reporting corruption +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +# Unmount to kill the healer +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full + +status=0 +exit
tests/xfs/1884.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 1884 +ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning
Re: [PATCH 06/14] xfs: test xfs_healer can fix a filesystem
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that
xfs_healercan actually fix an injected metadata corruption.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 111 lines
tests/xfs/1884 | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1884.out | 2 + 2 files changed, 91 insertions(+) create mode 100755 tests/xfs/1884 create mode 100644 tests/xfs/1884.outtests/xfs/1884 new file +89
-0@@ -0,0 +1,89 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1884 +# +# Ensure that autonomous self healing fixes the filesystem correctly. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --repair + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err + +# List the dirents of /victimdir to see if it stops reporting corruption +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +# Unmount to kill the healer +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full + +status=0 +exittests/xfs/1884.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 1884 +ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning
PATCH 7/14 xfs: test xfs_healer can report file I/O errors
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that xfs_healer can actually report file I/O errors.
--- tests/xfs/1896 | 210 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1896.out | 21 +++++ 2 files changed, 231 insertions(+) create mode 100755 tests/xfs/1896 create mode 100644 tests/xfs/1896.out
tests/xfs/1896 new file +210 -0
@@ -0,0 +1,210 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1896 +# +# Check that xfs_healer can report file IO errors. + +. ./common/preamble +_begin_fstest auto quick scrub eio selfhealing + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -f $tmp.* + _dmerror_cleanup +} + +# Import common functions. +. ./common/fuzzy +. ./common/filter +. ./common/dmerror +. ./common/systemd + +_require_scratch +_require_scrub +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_dm_target error +_require_no_xfs_always_cow # no out of place writes + +# Ignore everything from the healer except for the four IO error log messages. +# Strip out file handle and range information because the blocksize can vary. +# Writeback and readahead can trigger multiple error messages due to retries, +# hence the uniq. +filter_healer_errors() { + _filter_scratch | \ + grep -E '(buffered|directio)' | \ + sed \ + -e 's/ino [0-9]*/ino NUM/g' \ + -e 's/gen 0x[0-9a-f]*/gen NUM/g' \ + -e 's/pos [0-9]*/pos NUM/g' \ + -e 's/len [0-9]*/len NUM/g' \ + -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' | \ + sort | \ + uniq +} + +_scratch_mkfs >> $seqres.full + +# +# The dm-error map added by this test doesn't work on zoned devices because +# table sizes need to be aligned to the zone size, and even for zoned on +# conventional this test will get confused because of the internal RT device. +# +# That check requires a mounted file system, so do a dummy mount before setting +# up DM. +# +_scratch_mount +_require_xfs_scratch_non_zoned +_require_xfs_healer $SCRATCH_MNT +_scratch_unmount + +_dmerror_init +_dmerror_mount >> $seqres.full 2>&1 + +# Write a file with 4 file blocks worth of data, figure out the LBA to target +victim=$SCRATCH_MNT/a +file_blksz=$(_get_file_block_size $SCRATCH_MNT) +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full +unset errordev + +awk_len_prog='{print $6}' +if _xfs_is_realtime_file $victim; then + if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then + awk_len_prog='{print $4}' + fi + errordev="RT" +fi +bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")" +echo "$errordev:$bmap_str" >> $seqres.full + +phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')" +len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")" + +fs_blksz=$(_get_block_size $SCRATCH_MNT) +echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full +kernel_sectors_per_fs_block=$((fs_blksz / 512)) + +# Did we get at least 4 fs blocks worth of extent? +min_len_sectors=$(( 4 * kernel_sectors_per_fs_block )) +test "$len" -lt $min_len_sectors && \ + _fail "could not format a long enough extent on an empty fs??" + +phys_start=$(echo "$phys" | sed -e 's/\.\..*//g') + +echo "$errordev:$phys:$len:$fs_blksz:$phys_start" >> $seqres.full +echo "victim file:" >> $seqres.full +od -tx1 -Ad -c $victim >> $seqres.full + +# Set the dmerror table so that all IO will pass through. +_dmerror_reset_table + +cat >> $seqres.full << ENDL +dmerror before: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# All sector numbers that we feed to the kernel must be in units of 512b, but +# they also must be aligned to the device's logical block size. +logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` +kernel_sectors_per_device_lba=$((logical_block_size / 512)) + +# Mark as bad one of the device LBAs in the middle of the extent. Target the +# second LBA of the third block of the four-block file extent that we allocated +# earlier, but without overflowing into the fourth file block. +bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) )) +bad_len=$kernel_sectors_per_device_lba +if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then + bad_sector=$((bad_sector + kernel_sectors_per_device_lba)) +fi +if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then + echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size" +fi + +# Remount to flush the page cache, start the healer, and make the LBA bad +_dmerror_unmount +_dmerror_mount + +_scratch_invoke_xfs_healer "$tmp.healer" + +_dmerror_mark_range_bad $bad_sector $bad_len $errordev + +cat >> $seqres.full << ENDL +dmerror after marking bad: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +_dmerror_load_error_table + +# See if buffered reads pick it up +echo "Try buffered read" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Now mark the bad range good so that unmount won't fail due to IO errors. +echo "Fix device" +_dmerror_mark_range_good $bad_sector $bad_len $errordev +_dmerror_load_error_table + +cat >> $seqres.full << ENDL +dmerror after marking good: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# Unmount filesystem to start fresh +echo "Kill healer" +_scratch_kill_xfs_healer _dmerror_unmount +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer_errors + +# Start the healer again so that can verify that the errors don't persist after +# we flip back to the good dm table. +echo "Remount and restart healer" +_dmerror_mount +_scratch_invoke_xfs_healer "$tmp.healer" + +# See if buffered reads pick it up +echo "Try buffered read again" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read again" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write again" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write again" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Unmount fs to kill healer, then wait for it to finish +echo "Kill healer again" +_scratch_kill_xfs_healer _dmerror_unmount +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer_errors + +# success, all done +status=0 +exit
tests/xfs/1896.out new file +21 -0
@@ -0,0 +1,21 @@ +QA output created by 1896 +Try buffered read +pread: Input/output error +Try directio read +pread: Input/output error +Try directio write +pwrite: Input/output error +Try buffered write +fsync: Input/output error +Fix device +Kill healer +VICTIM pos NUM len NUM: buffered_read: Input/output error +VICTIM pos NUM len NUM: buffered_write: Input/output error +VICTIM pos NUM len NUM: directio_read: Input/output error +VICTIM pos NUM len NUM: directio_write: Input/output error +Remount and restart healer +Try buffered read again +Try directio read again +Try directio write again +Try buffered write again +Kill healer again
Re: [PATCH 07/14] xfs: test xfs_healer can report file I/O errors
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that
xfs_healercan actually report file I/O errors.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 251 lines
tests/xfs/1896 | 210 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1896.out | 21 +++++ 2 files changed, 231 insertions(+) create mode 100755 tests/xfs/1896 create mode 100644 tests/xfs/1896.outtests/xfs/1896 new file +210
-0@@ -0,0 +1,210 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1896 +# +# Check that xfs_healer can report file IO errors. + +. ./common/preamble +_begin_fstest auto quick scrub eio selfhealing + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -f $tmp.* + _dmerror_cleanup +} + +# Import common functions. +. ./common/fuzzy +. ./common/filter +. ./common/dmerror +. ./common/systemd + +_require_scratch +_require_scrub +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_dm_target error +_require_no_xfs_always_cow # no out of place writes + +# Ignore everything from the healer except for the four IO error log messages. +# Strip out file handle and range information because the blocksize can vary. +# Writeback and readahead can trigger multiple error messages due to retries, +# hence the uniq. +filter_healer_errors() { + _filter_scratch | \ + grep -E '(buffered|directio)' | \ + sed \ + -e 's/ino [0-9]*/ino NUM/g' \ + -e 's/gen 0x[0-9a-f]*/gen NUM/g' \ + -e 's/pos [0-9]*/pos NUM/g' \ + -e 's/len [0-9]*/len NUM/g' \ + -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' | \ + sort | \ + uniq +} + +_scratch_mkfs >> $seqres.full + +# +# The dm-error map added by this test doesn't work on zoned devices because +# table sizes need to be aligned to the zone size, and even for zoned on +# conventional this test will get confused because of the internal RT device. +# +# That check requires a mounted file system, so do a dummy mount before setting +# up DM. +# +_scratch_mount +_require_xfs_scratch_non_zoned +_require_xfs_healer $SCRATCH_MNT +_scratch_unmount + +_dmerror_init +_dmerror_mount >> $seqres.full 2>&1 + +# Write a file with 4 file blocks worth of data, figure out the LBA to target +victim=$SCRATCH_MNT/a +file_blksz=$(_get_file_block_size $SCRATCH_MNT) +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full +unset errordev + +awk_len_prog='{print $6}' +if _xfs_is_realtime_file $victim; then + if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then + awk_len_prog='{print $4}' + fi + errordev="RT" +fi +bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")" +echo "$errordev:$bmap_str" >> $seqres.full + +phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')" +len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")" + +fs_blksz=$(_get_block_size $SCRATCH_MNT) +echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full +kernel_sectors_per_fs_block=$((fs_blksz / 512)) + +# Did we get at least 4 fs blocks worth of extent? +min_len_sectors=$(( 4 * kernel_sectors_per_fs_block )) +test "$len" -lt $min_len_sectors && \ + _fail "could not format a long enough extent on an empty fs??" + +phys_start=$(echo "$phys" | sed -e 's/\.\..*//g') + +echo "$errordev:$phys:$len:$fs_blksz:$phys_start" >> $seqres.full +echo "victim file:" >> $seqres.full +od -tx1 -Ad -c $victim >> $seqres.full + +# Set the dmerror table so that all IO will pass through. +_dmerror_reset_table + +cat >> $seqres.full << ENDL +dmerror before: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# All sector numbers that we feed to the kernel must be in units of 512b, but +# they also must be aligned to the device's logical block size. +logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` +kernel_sectors_per_device_lba=$((logical_block_size / 512)) + +# Mark as bad one of the device LBAs in the middle of the extent. Target the +# second LBA of the third block of the four-block file extent that we allocated +# earlier, but without overflowing into the fourth file block. +bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) )) +bad_len=$kernel_sectors_per_device_lba +if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then + bad_sector=$((bad_sector + kernel_sectors_per_device_lba)) +fi +if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then + echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size" +fi + +# Remount to flush the page cache, start the healer, and make the LBA bad +_dmerror_unmount +_dmerror_mount + +_scratch_invoke_xfs_healer "$tmp.healer" + +_dmerror_mark_range_bad $bad_sector $bad_len $errordev + +cat >> $seqres.full << ENDL +dmerror after marking bad: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +_dmerror_load_error_table + +# See if buffered reads pick it up +echo "Try buffered read" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Now mark the bad range good so that unmount won't fail due to IO errors. +echo "Fix device" +_dmerror_mark_range_good $bad_sector $bad_len $errordev +_dmerror_load_error_table + +cat >> $seqres.full << ENDL +dmerror after marking good: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# Unmount filesystem to start fresh +echo "Kill healer" +_scratch_kill_xfs_healer _dmerror_unmount +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer_errors + +# Start the healer again so that can verify that the errors don't persist after +# we flip back to the good dm table. +echo "Remount and restart healer" +_dmerror_mount +_scratch_invoke_xfs_healer "$tmp.healer" + +# See if buffered reads pick it up +echo "Try buffered read again" +$XFS_IO_PROG -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio reads pick it up +echo "Try directio read again" +$XFS_IO_PROG -d -c "pread 0 $((4 * file_blksz))" $victim >> $seqres.full + +# See if directio writes pick it up +echo "Try directio write again" +$XFS_IO_PROG -d -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# See if buffered writes pick it up +echo "Try buffered write again" +$XFS_IO_PROG -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c fsync $victim >> $seqres.full + +# Unmount fs to kill healer, then wait for it to finish +echo "Kill healer again" +_scratch_kill_xfs_healer _dmerror_unmount +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer_errors + +# success, all done +status=0 +exittests/xfs/1896.out new file +21
-0@@ -0,0 +1,21 @@ +QA output created by 1896 +Try buffered read +pread: Input/output error +Try directio read +pread: Input/output error +Try directio write +pwrite: Input/output error +Try buffered write +fsync: Input/output error +Fix device +Kill healer +VICTIM pos NUM len NUM: buffered_read: Input/output error +VICTIM pos NUM len NUM: buffered_write: Input/output error +VICTIM pos NUM len NUM: directio_read: Input/output error +VICTIM pos NUM len NUM: directio_write: Input/output error +Remount and restart healer +Try buffered read again +Try directio read again +Try directio write again +Try buffered write again +Kill healer again
PATCH 8/14 xfs: test xfs_healer can report file media errors
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that xfs_healer can actually report media errors as found by the kernel.
--- tests/xfs/1897 | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1897.out | 7 ++ 2 files changed, 179 insertions(+) create mode 100755 tests/xfs/1897 create mode 100755 tests/xfs/1897.out
tests/xfs/1897 new file +172 -0
@@ -0,0 +1,172 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1897 +# +# Check that xfs_healer can report media errors. + +. ./common/preamble +_begin_fstest auto quick scrub eio selfhealing + +_cleanup() +{ + cd / + rm -f $tmp.* + _dmerror_cleanup +} + +. ./common/fuzzy +. ./common/filter +. ./common/dmerror +. ./common/systemd + +_require_scratch +_require_scrub +_require_dm_target error +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_xfs_io_command verifymedia + +filter_healer() { + _filter_scratch | \ + grep -E '(media failed|media error)' | \ + sed \ + -e 's/datadev/DEVICE/g' \ + -e 's/rtdev/DEVICE/g' \ + -e 's/ino [0-9]*/ino NUM/g' \ + -e 's/gen 0x[0-9a-f]*/gen NUM/g' \ + -e 's/pos [0-9]*/pos NUM/g' \ + -e 's/len [0-9]*/len NUM/g' \ + -e 's/0x[0-9a-f]*/NUM/g' \ + -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' +} + +filter_verify() { + sed -e 's/\([a-z]*dev\): verify error at offset \([0-9]*\) length \([0-9]*\)/DEVICE: verify error at offset XXX length XXX/g' +} + +_scratch_mkfs >> $seqres.full + +# The dm-error map added by this test doesn't work on zoned devices because +# table sizes need to be aligned to the zone size, and even for zoned on +# conventional this test will get confused because of the internal RT device. +# +# That check requires a mounted file system, so do a dummy mount before setting +# up DM. +_scratch_mount +_require_xfs_scratch_non_zoned +_require_xfs_healer $SCRATCH_MNT +_scratch_unmount + +_dmerror_init +_dmerror_mount + +# Write a file with 4 file blocks worth of data, figure out the LBA to target +victim=$SCRATCH_MNT/a +file_blksz=$(_get_file_block_size $SCRATCH_MNT) +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full +unset errordev +verifymediadev="-d" + +awk_len_prog='{print $6}' +if _xfs_is_realtime_file $victim; then + if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then + awk_len_prog='{print $4}' + fi + errordev="RT" + verifymediadev="-r" +fi +bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")" +echo "$errordev:$bmap_str" >> $seqres.full + +phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')" +len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")" + +fs_blksz=$(_get_block_size $SCRATCH_MNT) +echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full +kernel_sectors_per_fs_block=$((fs_blksz / 512)) + +# Did we get at least 4 fs blocks worth of extent? +min_len_sectors=$(( 4 * kernel_sectors_per_fs_block )) +test "$len" -lt $min_len_sectors && \ + _fail "could not format a long enough extent on an empty fs??" + +phys_start=$(echo "$phys" | sed -e 's/\.\..*//g') + +echo "$errordev:$phys:$len:$fs_blksz:$phys_start" >> $seqres.full +echo "victim file:" >> $seqres.full +od -tx1 -Ad -c $victim >> $seqres.full + +# Set the dmerror table so that all IO will pass through. +_dmerror_reset_table + +cat >> $seqres.full << ENDL +dmerror before: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# All sector numbers that we feed to the kernel must be in units of 512b, but +# they also must be aligned to the device's logical block size. +logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` +kernel_sectors_per_device_lba=$((logical_block_size / 512)) + +# Pretend as bad one of the device LBAs in the middle of the extent. Target +# the second LBA of the third block of the four-block file extent that we +# allocated earlier, but without overflowing into the fourth file block. +bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) )) +bad_len=$kernel_sectors_per_device_lba +if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then + bad_sector=$((bad_sector + kernel_sectors_per_device_lba)) +fi +if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then + echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size" +fi +_dmerror_mark_range_bad $bad_sector $bad_len $errordev + +cat >> $seqres.full << ENDL +dmerror after marking bad: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +_dmerror_load_error_table + +echo "Simulate media error" +_scratch_invoke_xfs_healer "$tmp.healer" +echo "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" >> $seqres.full +$XFS_IO_PROG -x -c "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" $SCRATCH_MNT 2>&1 | filter_verify + +# Now mark the bad range good so that a retest shows no media failure. +_dmerror_mark_range_good $bad_sector $bad_len $errordev +_dmerror_load_error_table + +cat >> $seqres.full << ENDL +dmerror after marking good: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +echo "No more media error" +echo "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" >> $seqres.full +$XFS_IO_PROG -x -c "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" $SCRATCH_MNT >> $seqres.full + +# Unmount filesystem to start fresh +echo "Kill healer" +_scratch_kill_xfs_healer _dmerror_unmount + +# filesystems without rmap do not translate media errors to lost file ranges +# so fake the output +_xfs_has_feature "$SCRATCH_DEV" rmapbt || \ + echo "VICTIM pos 0 len 0: media failed" >> $tmp.healer + +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer + +# success, all done +status=0 +exit
tests/xfs/1897.out new file +7 -0
@@ -0,0 +1,7 @@ +QA output created by 1897 +Simulate media error +DEVICE: verify error at offset XXX length XXX: Input/output error +No more media error +Kill healer +SCRATCH_MNT DEVICE daddr NUM bbcount NUM: media error +VICTIM pos NUM len NUM: media failed
Re: [PATCH 08/14] xfs: test xfs_healer can report file media errors
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that
xfs_healercan actually report media errors as found by the kernel.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 199 lines
tests/xfs/1897 | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1897.out | 7 ++ 2 files changed, 179 insertions(+) create mode 100755 tests/xfs/1897 create mode 100755 tests/xfs/1897.outtests/xfs/1897 new file +172
-0@@ -0,0 +1,172 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1897 +# +# Check that xfs_healer can report media errors. + +. ./common/preamble +_begin_fstest auto quick scrub eio selfhealing + +_cleanup() +{ + cd / + rm -f $tmp.* + _dmerror_cleanup +} + +. ./common/fuzzy +. ./common/filter +. ./common/dmerror +. ./common/systemd + +_require_scratch +_require_scrub +_require_dm_target error +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_xfs_io_command verifymedia + +filter_healer() { + _filter_scratch | \ + grep -E '(media failed|media error)' | \ + sed \ + -e 's/datadev/DEVICE/g' \ + -e 's/rtdev/DEVICE/g' \ + -e 's/ino [0-9]*/ino NUM/g' \ + -e 's/gen 0x[0-9a-f]*/gen NUM/g' \ + -e 's/pos [0-9]*/pos NUM/g' \ + -e 's/len [0-9]*/len NUM/g' \ + -e 's/0x[0-9a-f]*/NUM/g' \ + -e 's|SCRATCH_MNT/a|VICTIM|g' \ + -e 's|SCRATCH_MNT ino NUM gen NUM|VICTIM|g' +} + +filter_verify() { + sed -e 's/\([a-z]*dev\): verify error at offset \([0-9]*\) length \([0-9]*\)/DEVICE: verify error at offset XXX length XXX/g' +} + +_scratch_mkfs >> $seqres.full + +# The dm-error map added by this test doesn't work on zoned devices because +# table sizes need to be aligned to the zone size, and even for zoned on +# conventional this test will get confused because of the internal RT device. +# +# That check requires a mounted file system, so do a dummy mount before setting +# up DM. +_scratch_mount +_require_xfs_scratch_non_zoned +_require_xfs_healer $SCRATCH_MNT +_scratch_unmount + +_dmerror_init +_dmerror_mount + +# Write a file with 4 file blocks worth of data, figure out the LBA to target +victim=$SCRATCH_MNT/a +file_blksz=$(_get_file_block_size $SCRATCH_MNT) +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full +unset errordev +verifymediadev="-d" + +awk_len_prog='{print $6}' +if _xfs_is_realtime_file $victim; then + if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then + awk_len_prog='{print $4}' + fi + errordev="RT" + verifymediadev="-r" +fi +bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")" +echo "$errordev:$bmap_str" >> $seqres.full + +phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')" +len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")" + +fs_blksz=$(_get_block_size $SCRATCH_MNT) +echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full +kernel_sectors_per_fs_block=$((fs_blksz / 512)) + +# Did we get at least 4 fs blocks worth of extent? +min_len_sectors=$(( 4 * kernel_sectors_per_fs_block )) +test "$len" -lt $min_len_sectors && \ + _fail "could not format a long enough extent on an empty fs??" + +phys_start=$(echo "$phys" | sed -e 's/\.\..*//g') + +echo "$errordev:$phys:$len:$fs_blksz:$phys_start" >> $seqres.full +echo "victim file:" >> $seqres.full +od -tx1 -Ad -c $victim >> $seqres.full + +# Set the dmerror table so that all IO will pass through. +_dmerror_reset_table + +cat >> $seqres.full << ENDL +dmerror before: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +# All sector numbers that we feed to the kernel must be in units of 512b, but +# they also must be aligned to the device's logical block size. +logical_block_size=`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` +kernel_sectors_per_device_lba=$((logical_block_size / 512)) + +# Pretend as bad one of the device LBAs in the middle of the extent. Target +# the second LBA of the third block of the four-block file extent that we +# allocated earlier, but without overflowing into the fourth file block. +bad_sector=$(( phys_start + (2 * kernel_sectors_per_fs_block) )) +bad_len=$kernel_sectors_per_device_lba +if (( kernel_sectors_per_device_lba < kernel_sectors_per_fs_block )); then + bad_sector=$((bad_sector + kernel_sectors_per_device_lba)) +fi +if (( (bad_sector % kernel_sectors_per_device_lba) != 0)); then + echo "bad_sector $bad_sector not congruent with device logical block size $logical_block_size" +fi +_dmerror_mark_range_bad $bad_sector $bad_len $errordev + +cat >> $seqres.full << ENDL +dmerror after marking bad: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +_dmerror_load_error_table + +echo "Simulate media error" +_scratch_invoke_xfs_healer "$tmp.healer" +echo "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" >> $seqres.full +$XFS_IO_PROG -x -c "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" $SCRATCH_MNT 2>&1 | filter_verify + +# Now mark the bad range good so that a retest shows no media failure. +_dmerror_mark_range_good $bad_sector $bad_len $errordev +_dmerror_load_error_table + +cat >> $seqres.full << ENDL +dmerror after marking good: +$DMERROR_TABLE +$DMERROR_RTTABLE +<end table> +ENDL + +echo "No more media error" +echo "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" >> $seqres.full +$XFS_IO_PROG -x -c "verifymedia $verifymediadev -R $((bad_sector * 512)) $(((bad_sector + bad_len) * 512))" $SCRATCH_MNT >> $seqres.full + +# Unmount filesystem to start fresh +echo "Kill healer" +_scratch_kill_xfs_healer _dmerror_unmount + +# filesystems without rmap do not translate media errors to lost file ranges +# so fake the output +_xfs_has_feature "$SCRATCH_DEV" rmapbt || \ + echo "VICTIM pos 0 len 0: media failed" >> $tmp.healer + +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer + +# success, all done +status=0 +exittests/xfs/1897.out new file +7
-0@@ -0,0 +1,7 @@ +QA output created by 1897 +Simulate media error +DEVICE: verify error at offset XXX length XXX: Input/output error +No more media error +Kill healer +SCRATCH_MNT DEVICE daddr NUM bbcount NUM: media error +VICTIM pos NUM len NUM: media failed
PATCH 9/14 xfs: test xfs_healer can report filesystem shutdowns
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that xfs_healer can actually report abnormal filesystem shutdowns.
--- tests/xfs/1898 | 37 +++++++++++++++++++++++++++++++++++++ tests/xfs/1898.out | 4 ++++ 2 files changed, 41 insertions(+) create mode 100755 tests/xfs/1898 create mode 100755 tests/xfs/1898.out
tests/xfs/1898 new file +37 -0
@@ -0,0 +1,37 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1898 +# +# Check that xfs_healer can report filesystem shutdowns. + +. ./common/preamble +_begin_fstest auto quick scrub eio selfhealing + +. ./common/fuzzy +. ./common/filter +. ./common/systemd + +_require_scratch_nocheck +_require_scrub +_require_command "$XFS_HEALER_PROG" "xfs_healer" + +_scratch_mkfs >> $seqres.full +_scratch_mount +_require_xfs_healer $SCRATCH_MNT +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 500k" -c "fsync" $victim >> $seqres.full + +echo "Start healer and shut down" +_scratch_invoke_xfs_healer "$tmp.healer" +_scratch_shutdown -f + +# Unmount filesystem to start fresh +echo "Kill healer" +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full +cat $tmp.healer | _filter_scratch | grep 'shut down' + +# success, all done +status=0 +exit
tests/xfs/1898.out new file +4 -0
@@ -0,0 +1,4 @@ +QA output created by 1898 +Start healer and shut down +Kill healer +SCRATCH_MNT: filesystem shut down due to forced unmount
Re: [PATCH 09/14] xfs: test xfs_healer can report filesystem shutdowns
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that
xfs_healercan actually report abnormal filesystem shutdowns.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 61 lines
tests/xfs/1898 | 37 +++++++++++++++++++++++++++++++++++++ tests/xfs/1898.out | 4 ++++ 2 files changed, 41 insertions(+) create mode 100755 tests/xfs/1898 create mode 100755 tests/xfs/1898.outtests/xfs/1898 new file +37
-0@@ -0,0 +1,37 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1898 +# +# Check that xfs_healer can report filesystem shutdowns. + +. ./common/preamble +_begin_fstest auto quick scrub eio selfhealing + +. ./common/fuzzy +. ./common/filter +. ./common/systemd + +_require_scratch_nocheck +_require_scrub +_require_command "$XFS_HEALER_PROG" "xfs_healer" + +_scratch_mkfs >> $seqres.full +_scratch_mount +_require_xfs_healer $SCRATCH_MNT +$XFS_IO_PROG -f -c "pwrite -S 0x58 0 500k" -c "fsync" $victim >> $seqres.full + +echo "Start healer and shut down" +_scratch_invoke_xfs_healer "$tmp.healer" +_scratch_shutdown -f + +# Unmount filesystem to start fresh +echo "Kill healer" +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full +cat $tmp.healer | _filter_scratch | grep 'shut down' + +# success, all done +status=0 +exittests/xfs/1898.out new file +4
-0@@ -0,0 +1,4 @@ +QA output created by 1898 +Start healer and shut down +Kill healer +SCRATCH_MNT: filesystem shut down due to forced unmount
PATCH 10/14 xfs: test xfs_healer can initiate full filesystem repairs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that when xfs_healer can't perform a spot repair, it will actually start up xfs_scrub to perform a full scan and repair.
--- tests/xfs/1899 | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1899.out | 3 + 2 files changed, 111 insertions(+) create mode 100755 tests/xfs/1899 create mode 100644 tests/xfs/1899.out
tests/xfs/1899 new file +108 -0
@@ -0,0 +1,108 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1899 +# +# Ensure that autonomous self healing works fixes the filesystem correctly +# even if the spot repair doesn't work and it falls back to a full fsck. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch +_require_systemd_unit_defined "[email protected]" + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +filter_healer() { + _filter_scratch | \ + grep 'Full repairs in progress' | \ + uniq +} + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' \ + -c 'path /a' \ + -c 'bmap -a' \ + -c 'ablock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' \ + >> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --repair + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err + +# Wait for the background fixer to finish +svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")" +_systemd_unit_wait "$svc" + +# List the dirents of /victimdir and parent pointers of /a to see if they both +# stop reporting corruption +(ls $SCRATCH_MNT/some/victimdir ; $XFS_IO_PROG -c 'parent') > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + (ls $SCRATCH_MNT/some/victimdir ; $XFS_IO_PROG -c 'parent') > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +# Unmount to kill the healer +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer + +status=0 +exit
tests/xfs/1899.out new file +3 -0
@@ -0,0 +1,3 @@ +QA output created by 1899 +ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning +SCRATCH_MNT: Full repairs in progress.
Re: [PATCH 10/14] xfs: test xfs_healer can initiate full filesystem repairs
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that when
xfs_healercan't perform a spot repair, it will actually start upxfs_scrubto perform a full scan and repair.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 131 lines
tests/xfs/1899 | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1899.out | 3 + 2 files changed, 111 insertions(+) create mode 100755 tests/xfs/1899 create mode 100644 tests/xfs/1899.outtests/xfs/1899 new file +108
-0@@ -0,0 +1,108 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1899 +# +# Ensure that autonomous self healing works fixes the filesystem correctly +# even if the spot repair doesn't work and it falls back to a full fsck. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch +_require_systemd_unit_defined "[email protected]" + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +filter_healer() { + _filter_scratch | \ + grep 'Full repairs in progress' | \ + uniq +} + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' \ + -c 'path /a' \ + -c 'bmap -a' \ + -c 'ablock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' \ + >> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --repair + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err + +# Wait for the background fixer to finish +svc="$(_xfs_scrub_svcname "$SCRATCH_MNT")" +_systemd_unit_wait "$svc" + +# List the dirents of /victimdir and parent pointers of /a to see if they both +# stop reporting corruption +(ls $SCRATCH_MNT/some/victimdir ; $XFS_IO_PROG -c 'parent') > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + (ls $SCRATCH_MNT/some/victimdir ; $XFS_IO_PROG -c 'parent') > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +# Unmount to kill the healer +_scratch_kill_xfs_healer +cat $tmp.healer >> $seqres.full +cat $tmp.healer | filter_healer + +status=0 +exittests/xfs/1899.out new file +3
-0@@ -0,0 +1,3 @@ +QA output created by 1899 +ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning +SCRATCH_MNT: Full repairs in progress.
PATCH 11/14 xfs: test xfs_healer can follow mount moves
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that when xfs_healer needs to reopen a filesystem to repair it, it can still find the filesystem even if it has been mount --move'd.
--- tests/xfs/1900 | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1900.out | 2 + 2 files changed, 117 insertions(+) create mode 100755 tests/xfs/1900 create mode 100755 tests/xfs/1900.out
tests/xfs/1900 new file +115 -0
@@ -0,0 +1,115 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1900 +# +# Ensure that autonomous self healing fixes the filesystem correctly even if +# the original mount has moved somewhere else. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_cleanup() +{ + command -v _kill_fsstress &>/dev/null && _kill_fsstress + cd / + rm -r -f $tmp.* + if [ -n "$new_dir" ]; then + _unmount "$new_dir" &>/dev/null + rm -rf "$new_dir" + fi +} + +_require_test +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --repair + +# Move the scratch filesystem to a completely different mountpoint so that +# we can test if the healer can find it again. +new_dir=$TEST_DIR/moocow +mkdir -p $new_dir +_mount --bind $SCRATCH_MNT $new_dir +_unmount $SCRATCH_MNT + +df -t xfs >> $seqres.full + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $new_dir/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err | _filter_test_dir +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err | _filter_test_dir + +# List the dirents of /victimdir to see if it stops reporting corruption +ls $new_dir/some/victimdir > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +new_dir_unmount() { + _unmount $new_dir +} + +# Unmount to kill the healer +_scratch_kill_xfs_healer new_dir_unmount +cat $tmp.healer >> $seqres.full + +status=0 +exit
tests/xfs/1900.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 1900 +ls: reading directory 'TEST_DIR/moocow/some/victimdir': Structure needs cleaning
Re: [PATCH 11/14] xfs: test xfs_healer can follow mount moves
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that when
xfs_healerneeds to reopen a filesystem to repair it, it can still find the filesystem even if it has been mount --move'd.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 137 lines
tests/xfs/1900 | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1900.out | 2 + 2 files changed, 117 insertions(+) create mode 100755 tests/xfs/1900 create mode 100755 tests/xfs/1900.outtests/xfs/1900 new file +115
-0@@ -0,0 +1,115 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1900 +# +# Ensure that autonomous self healing fixes the filesystem correctly even if +# the original mount has moved somewhere else. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_cleanup() +{ + command -v _kill_fsstress &>/dev/null && _kill_fsstress + cd / + rm -r -f $tmp.* + if [ -n "$new_dir" ]; then + _unmount "$new_dir" &>/dev/null + rm -rf "$new_dir" + fi +} + +_require_test +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full +_scratch_mount + +_scratch_invoke_xfs_healer "$tmp.healer" --repair + +# Move the scratch filesystem to a completely different mountpoint so that +# we can test if the healer can find it again. +new_dir=$TEST_DIR/moocow +mkdir -p $new_dir +_mount --bind $SCRATCH_MNT $new_dir +_unmount $SCRATCH_MNT + +df -t xfs >> $seqres.full + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $new_dir/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err | _filter_test_dir +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err | _filter_test_dir + +# List the dirents of /victimdir to see if it stops reporting corruption +ls $new_dir/some/victimdir > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +new_dir_unmount() { + _unmount $new_dir +} + +# Unmount to kill the healer +_scratch_kill_xfs_healer new_dir_unmount +cat $tmp.healer >> $seqres.full + +status=0 +exittests/xfs/1900.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 1900 +ls: reading directory 'TEST_DIR/moocow/some/victimdir': Structure needs cleaning
PATCH 12/14 xfs: test xfs_healer wont repair the wrong filesystem
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that when xfs_healer needs to reopen a filesystem to repair it, it won't latch on to another xfs filesystem that has been mounted atop the same mountpoint.
--- tests/xfs/1901 | 137 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1901.out | 2 + 2 files changed, 139 insertions(+) create mode 100755 tests/xfs/1901 create mode 100755 tests/xfs/1901.out
tests/xfs/1901 new file +137 -0
@@ -0,0 +1,137 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1901 +# +# Ensure that autonomous self healing won't fix the wrong filesystem if a +# snapshot of the original filesystem is now mounted on the same directory as +# the original. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_cleanup() +{ + command -v _kill_fsstress &>/dev/null && _kill_fsstress + cd / + rm -r -f $tmp.* + test -e "$mntpt" && _unmount "$mntpt" &>/dev/null + test -e "$mntpt" && _unmount "$mntpt" &>/dev/null + test -e "$loop1" && _destroy_loop_device "$loop1" + test -e "$loop2" && _destroy_loop_device "$loop2" + test -e "$testdir" && rm -r -f "$testdir" +} + +_require_test +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" + +testdir=$TEST_DIR/$seq +mntpt=$testdir/mount +disk1=$testdir/disk1 +disk2=$testdir/disk2 + +mkdir -p "$mntpt" +$XFS_IO_PROG -f -c "truncate 300m" $disk1 +$XFS_IO_PROG -f -c "truncate 300m" $disk2 +loop1="$(_create_loop_device "$disk1")" + +filter_mntpt() { + sed -e "s|$mntpt|MNTPT|g" +} + +_mkfs_dev "$loop1" >> $seqres.full +_mount "$loop1" "$mntpt" || _notrun "cannot mount victim filesystem" + +_xfs_has_feature $mntpt rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $mntpt parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $mntpt --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $mntpt set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$mntpt") +echo testdata > $mntpt/a +mkdir -p "$mntpt/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $mntpt/a $mntpt/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $mntpt/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Clone the fs, break the directory, remount filesystem +_unmount "$mntpt" + +cp --sparse=always "$disk1" "$disk2" || _fail "cannot copy disk1" +loop2="$(_create_loop_device_like_bdev "$disk2" "$loop1")" + +$XFS_DB_PROG "$loop1" -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full +_mount "$loop1" "$mntpt" || _fail "cannot mount broken fs" + +_invoke_xfs_healer "$mntpt" "$tmp.healer" --repair + +# Stop the healer process so that it can't read error events while we do some +# shenanigans. +test -n "$XFS_HEALER_PID" || _fail "nobody set XFS_HEALER_PID?" +kill -STOP $XFS_HEALER_PID + + +echo "LOG $XFS_HEALER_PID SO FAR:" >> $seqres.full +cat $tmp.healer >> $seqres.full + +# Access the broken directory to trigger a repair event, which will not yet be +# processed. +ls $mntpt/some/victimdir > /dev/null 2> $tmp.err +filter_mntpt < $tmp.err + +ps auxfww | grep xfs_healer >> $seqres.full + +echo "LOG AFTER TRYING TO POKE:" >> $seqres.full +cat $tmp.healer >> $seqres.full + +# Mount the clone filesystem to the same mountpoint so that the healer cannot +# actually reopen it to perform repairs. +_mount "$loop2" "$mntpt" -o nouuid || _fail "cannot mount decoy fs" + +grep -w xfs /proc/mounts >> $seqres.full + +# Continue the healer process so it can handle events now. Wait a few seconds +# while it fails to reopen disk1's mount point to repair things. +kill -CONT $XFS_HEALER_PID +sleep 2 + +new_dir_unmount() { + _unmount "$mntpt" + _unmount "$mntpt" +} + +# Unmount to kill the healer +_kill_xfs_healer new_dir_unmount +echo "LOG AFTER FAILURE" >> $seqres.full +cat $tmp.healer >> $seqres.full + +# Did the healer log complaints about not being able to reopen the mountpoint +# to enact repairs? +grep -q 'Stale file handle' $tmp.healer || \ + echo "Should have seen stale file handle complaints" + +status=0 +exit
tests/xfs/1901.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 1901 +ls: reading directory 'MNTPT/some/victimdir': Structure needs cleaning
Re: [PATCH 12/14] xfs: test xfs_healer wont repair the wrong filesystem
Quoted 8 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that when
xfs_healerneeds to reopen a filesystem to repair it, it won't latch on to another xfs filesystem that has been mounted atop the same mountpoint.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 159 lines
tests/xfs/1901 | 137 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1901.out | 2 + 2 files changed, 139 insertions(+) create mode 100755 tests/xfs/1901 create mode 100755 tests/xfs/1901.outtests/xfs/1901 new file +137
-0@@ -0,0 +1,137 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1901 +# +# Ensure that autonomous self healing won't fix the wrong filesystem if a +# snapshot of the original filesystem is now mounted on the same directory as +# the original. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_cleanup() +{ + command -v _kill_fsstress &>/dev/null && _kill_fsstress + cd / + rm -r -f $tmp.* + test -e "$mntpt" && _unmount "$mntpt" &>/dev/null + test -e "$mntpt" && _unmount "$mntpt" &>/dev/null + test -e "$loop1" && _destroy_loop_device "$loop1" + test -e "$loop2" && _destroy_loop_device "$loop2" + test -e "$testdir" && rm -r -f "$testdir" +} + +_require_test +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" + +testdir=$TEST_DIR/$seq +mntpt=$testdir/mount +disk1=$testdir/disk1 +disk2=$testdir/disk2 + +mkdir -p "$mntpt" +$XFS_IO_PROG -f -c "truncate 300m" $disk1 +$XFS_IO_PROG -f -c "truncate 300m" $disk2 +loop1="$(_create_loop_device "$disk1")" + +filter_mntpt() { + sed -e "s|$mntpt|MNTPT|g" +} + +_mkfs_dev "$loop1" >> $seqres.full +_mount "$loop1" "$mntpt" || _notrun "cannot mount victim filesystem" + +_xfs_has_feature $mntpt rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $mntpt parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $mntpt --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $mntpt set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$mntpt") +echo testdata > $mntpt/a +mkdir -p "$mntpt/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $mntpt/a $mntpt/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $mntpt/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Clone the fs, break the directory, remount filesystem +_unmount "$mntpt" + +cp --sparse=always "$disk1" "$disk2" || _fail "cannot copy disk1" +loop2="$(_create_loop_device_like_bdev "$disk2" "$loop1")" + +$XFS_DB_PROG "$loop1" -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full +_mount "$loop1" "$mntpt" || _fail "cannot mount broken fs" + +_invoke_xfs_healer "$mntpt" "$tmp.healer" --repair + +# Stop the healer process so that it can't read error events while we do some +# shenanigans. +test -n "$XFS_HEALER_PID" || _fail "nobody set XFS_HEALER_PID?" +kill -STOP $XFS_HEALER_PID + + +echo "LOG $XFS_HEALER_PID SO FAR:" >> $seqres.full +cat $tmp.healer >> $seqres.full + +# Access the broken directory to trigger a repair event, which will not yet be +# processed. +ls $mntpt/some/victimdir > /dev/null 2> $tmp.err +filter_mntpt < $tmp.err + +ps auxfww | grep xfs_healer >> $seqres.full + +echo "LOG AFTER TRYING TO POKE:" >> $seqres.full +cat $tmp.healer >> $seqres.full + +# Mount the clone filesystem to the same mountpoint so that the healer cannot +# actually reopen it to perform repairs. +_mount "$loop2" "$mntpt" -o nouuid || _fail "cannot mount decoy fs" + +grep -w xfs /proc/mounts >> $seqres.full + +# Continue the healer process so it can handle events now. Wait a few seconds +# while it fails to reopen disk1's mount point to repair things. +kill -CONT $XFS_HEALER_PID +sleep 2 + +new_dir_unmount() { + _unmount "$mntpt" + _unmount "$mntpt" +} + +# Unmount to kill the healer +_kill_xfs_healer new_dir_unmount +echo "LOG AFTER FAILURE" >> $seqres.full +cat $tmp.healer >> $seqres.full + +# Did the healer log complaints about not being able to reopen the mountpoint +# to enact repairs? +grep -q 'Stale file handle' $tmp.healer || \ + echo "Should have seen stale file handle complaints" + +status=0 +exittests/xfs/1901.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 1901 +ls: reading directory 'MNTPT/some/victimdir': Structure needs cleaning
PATCH 13/14 xfs: test xfs_healer background service
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that when xfs_healer can monitor and repair filesystems when it's running as a systemd service, which is the intended usage model.
--- tests/xfs/1902 | 152 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1902.out | 2 + 2 files changed, 154 insertions(+) create mode 100755 tests/xfs/1902 create mode 100755 tests/xfs/1902.out
tests/xfs/1902 new file +152 -0
@@ -0,0 +1,152 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1902 +# +# Ensure that autonomous self healing fixes the filesystem correctly when +# running in a systemd service +# +# unreliable_in_parallel: this test runs the xfs_healer systemd service, which +# cannot be isolated to a specific testcase with the way check-parallel is +# implemented. +# +. ./common/preamble +_begin_fstest auto selfhealing unreliable_in_parallel + +_cleanup() +{ + cd / + if [ -n "$new_svcfile" ]; then + rm -f "$new_svcfile" + systemctl daemon-reload + fi + rm -r -f $tmp.* +} + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_require_systemd_is_running +_require_systemd_unit_defined [email protected] +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full + +# Find the existing xfs_healer@ service definition, figure out where we're +# going to land our test-specific override +orig_svcfile="$(_systemd_unit_path "[email protected]")" +test -f "$orig_svcfile" || \ + _notrun "cannot find xfs_healer@ service file" + +new_svcdir="$(_systemd_runtime_dir)" +test -d "$new_svcdir" || \ + _notrun "cannot find runtime systemd service dir" + +# We need to make some local mods to the xfs_healer@ service definition +# so we fork it and create a new service just for this test. +new_healer_template="[email protected]" +new_healer_svc="$(_systemd_service_unit_path "$new_healer_template" "$SCRATCH_MNT")" +_systemd_unit_status "$new_healer_svc" 2>&1 | \ + grep -E -q '(could not be found|Loaded: not-found)' || \ + _notrun "systemd service \"$new_healer_svc\" found, will not mess with this" + +new_svcfile="$new_svcdir/$new_healer_template" +cp "$orig_svcfile" "$new_svcfile" + +# Pick up all the CLI args except for --repair and --no-autofsck because we're +# going to force it to --autofsck below +execargs="$(grep '^ExecStart=' $new_svcfile | \ + sed -e 's/^ExecStart=\S*//g' \ + -e 's/--no-autofsck//g' \ + -e 's/--repair//g')" +sed -e '/ExecStart=/d' -e '/BindPaths=/d' -e '/ExecCondition=/d' -i $new_svcfile +cat >> "$new_svcfile" << ENDL + +[Service] +ExecCondition=$XFS_HEALER_PROG --supported %f +ExecStart=$XFS_HEALER_PROG $execargs +ENDL +_systemd_reload + +# Emit the results of our editing to the full log. +systemctl cat "$new_healer_svc" >> $seqres.full + +# Remount, with service activation +_scratch_mount + +old_healer_svc="$(_xfs_healer_svcname "$SCRATCH_MNT")" +_systemd_unit_stop "$old_healer_svc" &>> $seqres.full +_systemd_unit_start "$new_healer_svc" &>> $seqres.full + +_systemd_unit_status "$new_healer_svc" 2>&1 | grep -q 'Active: active' || \ + echo "systemd service \"$new_healer_svc\" not running??" + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err + +# List the dirents of /victimdir to see if it stops reporting corruption +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +# Unmount to kill the healer +_scratch_kill_xfs_healer +journalctl -u "$new_healer_svc" >> $seqres.full + +status=0 +exit
tests/xfs/1902.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 1902 +ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning
Re: [PATCH 13/14] xfs: test xfs_healer background service
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that when
xfs_healercan monitor and repair filesystems when it's running as a systemd service, which is the intended usage model.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 174 lines
tests/xfs/1902 | 152 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1902.out | 2 + 2 files changed, 154 insertions(+) create mode 100755 tests/xfs/1902 create mode 100755 tests/xfs/1902.outtests/xfs/1902 new file +152
-0@@ -0,0 +1,152 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024-2026 Oracle. All Rights Reserved. +# +# FS QA Test 1902 +# +# Ensure that autonomous self healing fixes the filesystem correctly when +# running in a systemd service +# +# unreliable_in_parallel: this test runs the xfs_healer systemd service, which +# cannot be isolated to a specific testcase with the way check-parallel is +# implemented. +# +. ./common/preamble +_begin_fstest auto selfhealing unreliable_in_parallel + +_cleanup() +{ + cd / + if [ -n "$new_svcfile" ]; then + rm -f "$new_svcfile" + systemctl daemon-reload + fi + rm -r -f $tmp.* +} + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +_require_systemd_is_running +_require_systemd_unit_defined [email protected] +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full + +# Find the existing xfs_healer@ service definition, figure out where we're +# going to land our test-specific override +orig_svcfile="$(_systemd_unit_path "[email protected]")" +test -f "$orig_svcfile" || \ + _notrun "cannot find xfs_healer@ service file" + +new_svcdir="$(_systemd_runtime_dir)" +test -d "$new_svcdir" || \ + _notrun "cannot find runtime systemd service dir" + +# We need to make some local mods to the xfs_healer@ service definition +# so we fork it and create a new service just for this test. +new_healer_template="[email protected]" +new_healer_svc="$(_systemd_service_unit_path "$new_healer_template" "$SCRATCH_MNT")" +_systemd_unit_status "$new_healer_svc" 2>&1 | \ + grep -E -q '(could not be found|Loaded: not-found)' || \ + _notrun "systemd service \"$new_healer_svc\" found, will not mess with this" + +new_svcfile="$new_svcdir/$new_healer_template" +cp "$orig_svcfile" "$new_svcfile" + +# Pick up all the CLI args except for --repair and --no-autofsck because we're +# going to force it to --autofsck below +execargs="$(grep '^ExecStart=' $new_svcfile | \ + sed -e 's/^ExecStart=\S*//g' \ + -e 's/--no-autofsck//g' \ + -e 's/--repair//g')" +sed -e '/ExecStart=/d' -e '/BindPaths=/d' -e '/ExecCondition=/d' -i $new_svcfile +cat >> "$new_svcfile" << ENDL + +[Service] +ExecCondition=$XFS_HEALER_PROG --supported %f +ExecStart=$XFS_HEALER_PROG $execargs +ENDL +_systemd_reload + +# Emit the results of our editing to the full log. +systemctl cat "$new_healer_svc" >> $seqres.full + +# Remount, with service activation +_scratch_mount + +old_healer_svc="$(_xfs_healer_svcname "$SCRATCH_MNT")" +_systemd_unit_stop "$old_healer_svc" &>> $seqres.full +_systemd_unit_start "$new_healer_svc" &>> $seqres.full + +_systemd_unit_status "$new_healer_svc" 2>&1 | grep -q 'Active: active' || \ + echo "systemd service \"$new_healer_svc\" not running??" + +# Access the broken directory to trigger a repair, then poll the directory +# for 5 seconds to see if it gets fixed without us needing to intervene. +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +_filter_scratch < $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "try $try no longer saw corruption or gave up" >> $seqres.full +_filter_scratch < $tmp.err + +# List the dirents of /victimdir to see if it stops reporting corruption +ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err +try=0 +while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) +done +echo "retry $try no longer saw corruption or gave up" >> $seqres.full + +# Unmount to kill the healer +_scratch_kill_xfs_healer +journalctl -u "$new_healer_svc" >> $seqres.full + +status=0 +exittests/xfs/1902.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 1902 +ls: reading directory 'SCRATCH_MNT/some/victimdir': Structure needs cleaning
PATCH 14/14 xfs: test xfs_healer startup service
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that xfs_healer_start can actually start up xfs_healer service instances when a filesystem is mounted.
--- tests/xfs/1903 | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1903.out | 6 +++ 2 files changed, 130 insertions(+) create mode 100755 tests/xfs/1903 create mode 100644 tests/xfs/1903.out
tests/xfs/1903 new file +124 -0
@@ -0,0 +1,124 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1903 +# +# Check that the xfs_healer startup service starts the per-mount xfs_healer +# service for the scratch filesystem. IOWs, this is basic testing for the +# xfs_healer systemd background services. +# + +# unreliable_in_parallel: this appears to try to run healer services on all +# mounted filesystems - that's a problem when there are a hundred other test +# filesystems mounted running other tests... + +. ./common/preamble +_begin_fstest auto selfhealing unreliable_in_parallel + +_cleanup() +{ + cd / + test -n "$new_healerstart_svc" && + _systemd_unit_stop "$new_healerstart_svc" + test -n "$was_masked" && \ + _systemd_unit_mask "$healer_svc" &>> $seqres.full + if [ -n "$new_svcfile" ]; then + rm -f "$new_svcfile" + systemctl daemon-reload + fi + rm -r -f $tmp.* +} + +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/systemd + +_require_systemd_is_running +_require_systemd_unit_defined [email protected] +_require_systemd_unit_defined xfs_healer_start.service +_require_scratch +_require_scrub +_require_xfs_io_command "scrub" +_require_xfs_spaceman_command "health" +_require_populate_commands +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command $ATTR_PROG "attr" + +_xfs_skip_online_rebuild +_xfs_skip_offline_rebuild + +orig_svcfile="$(_systemd_unit_path "xfs_healer_start.service")" +test -f "$orig_svcfile" || \ + _notrun "cannot find xfs_healer_start service file" + +new_svcdir="$(_systemd_runtime_dir)" +test -d "$new_svcdir" || \ + _notrun "cannot find runtime systemd service dir" + +# We need to make some local mods to the xfs_healer_start service definition +# so we fork it and create a new service just for this test. +new_healerstart_svc="xfs_healer_start_fstest.service" +_systemd_unit_status "$new_healerstart_svc" 2>&1 | \ + grep -E -q '(could not be found|Loaded: not-found)' || \ + _notrun "systemd service \"$new_healerstart_svc\" found, will not mess with this" + +find_healer_trace() { + local path="$1" + + sleep 2 # wait for delays in startup + $XFS_HEALER_PROG --supported "$path" 2>&1 | grep -q 'already running' || \ + echo "cannot find evidence that xfs_healer is running for $path" +} + +echo "Format and populate" +_scratch_mkfs >> $seqres.full +_scratch_mount +_require_xfs_healer $SCRATCH_MNT + +# Configure the filesystem for background checks of the filesystem. +$ATTR_PROG -R -s xfs:autofsck -V check $SCRATCH_MNT >> $seqres.full + +was_masked= +healer_svc="$(_xfs_healer_svcname "$SCRATCH_MNT")" + +# Preserve the xfs_healer@ mask state -- we don't want this permanently +# changing global state. +if _systemd_unit_masked "$healer_svc"; then + _systemd_unit_unmask "$healer_svc" &>> $seqres.full + was_masked=1 +fi + +echo "Start healer on scratch FS" +_systemd_unit_start "$healer_svc" +find_healer_trace "$SCRATCH_MNT" +_systemd_unit_stop "$healer_svc" + +new_svcfile="$new_svcdir/$new_healerstart_svc" +cp "$orig_svcfile" "$new_svcfile" + +sed -e '/ExecStart=/d' -e '/BindPaths=/d' -e '/ExecCondition=/d' -i $new_svcfile +cat >> "$new_svcfile" << ENDL +[Service] +ExecCondition=$XFS_HEALER_START_PROG --supported +ExecStart=$XFS_HEALER_START_PROG +ENDL +_systemd_reload + +# Emit the results of our editing to the full log. +systemctl cat "$new_healerstart_svc" >> $seqres.full + +echo "Start healer for everything" +_systemd_unit_start "$new_healerstart_svc" +find_healer_trace "$SCRATCH_MNT" + +echo "Restart healer for scratch FS" +_scratch_cycle_mount +find_healer_trace "$SCRATCH_MNT" + +echo "Healer testing done" | tee -a $seqres.full + +# success, all done +status=0 +exit
tests/xfs/1903.out new file +6 -0
@@ -0,0 +1,6 @@ +QA output created by 1903 +Format and populate +Start healer on scratch FS +Start healer for everything +Restart healer for scratch FS +Healer testing done
Re: [PATCH 14/14] xfs: test xfs_healer startup service
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that
xfs_healer_startcan actually start upxfs_healerservice instances when a filesystem is mounted.Signed-off-by "Darrick J. Wong" <[email protected]>---
Tests and looks good to me,
Quoted 150 lines
tests/xfs/1903 | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1903.out | 6 +++ 2 files changed, 130 insertions(+) create mode 100755 tests/xfs/1903 create mode 100644 tests/xfs/1903.outtests/xfs/1903 new file +124
-0@@ -0,0 +1,124 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Oracle. All Rights Reserved. +# +# FS QA Test No. 1903 +# +# Check that the xfs_healer startup service starts the per-mount xfs_healer +# service for the scratch filesystem. IOWs, this is basic testing for the +# xfs_healer systemd background services. +# + +# unreliable_in_parallel: this appears to try to run healer services on all +# mounted filesystems - that's a problem when there are a hundred other test +# filesystems mounted running other tests... + +. ./common/preamble +_begin_fstest auto selfhealing unreliable_in_parallel + +_cleanup() +{ + cd / + test -n "$new_healerstart_svc" && + _systemd_unit_stop "$new_healerstart_svc" + test -n "$was_masked" && \ + _systemd_unit_mask "$healer_svc" &>> $seqres.full + if [ -n "$new_svcfile" ]; then + rm -f "$new_svcfile" + systemctl daemon-reload + fi + rm -r -f $tmp.* +} + +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/systemd + +_require_systemd_is_running +_require_systemd_unit_defined [email protected] +_require_systemd_unit_defined xfs_healer_start.service +_require_scratch +_require_scrub +_require_xfs_io_command "scrub" +_require_xfs_spaceman_command "health" +_require_populate_commands +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command $ATTR_PROG "attr" + +_xfs_skip_online_rebuild +_xfs_skip_offline_rebuild + +orig_svcfile="$(_systemd_unit_path "xfs_healer_start.service")" +test -f "$orig_svcfile" || \ + _notrun "cannot find xfs_healer_start service file" + +new_svcdir="$(_systemd_runtime_dir)" +test -d "$new_svcdir" || \ + _notrun "cannot find runtime systemd service dir" + +# We need to make some local mods to the xfs_healer_start service definition +# so we fork it and create a new service just for this test. +new_healerstart_svc="xfs_healer_start_fstest.service" +_systemd_unit_status "$new_healerstart_svc" 2>&1 | \ + grep -E -q '(could not be found|Loaded: not-found)' || \ + _notrun "systemd service \"$new_healerstart_svc\" found, will not mess with this" + +find_healer_trace() { + local path="$1" + + sleep 2 # wait for delays in startup + $XFS_HEALER_PROG --supported "$path" 2>&1 | grep -q 'already running' || \ + echo "cannot find evidence that xfs_healer is running for $path" +} + +echo "Format and populate" +_scratch_mkfs >> $seqres.full +_scratch_mount +_require_xfs_healer $SCRATCH_MNT + +# Configure the filesystem for background checks of the filesystem. +$ATTR_PROG -R -s xfs:autofsck -V check $SCRATCH_MNT >> $seqres.full + +was_masked= +healer_svc="$(_xfs_healer_svcname "$SCRATCH_MNT")" + +# Preserve the xfs_healer@ mask state -- we don't want this permanently +# changing global state. +if _systemd_unit_masked "$healer_svc"; then + _systemd_unit_unmask "$healer_svc" &>> $seqres.full + was_masked=1 +fi + +echo "Start healer on scratch FS" +_systemd_unit_start "$healer_svc" +find_healer_trace "$SCRATCH_MNT" +_systemd_unit_stop "$healer_svc" + +new_svcfile="$new_svcdir/$new_healerstart_svc" +cp "$orig_svcfile" "$new_svcfile" + +sed -e '/ExecStart=/d' -e '/BindPaths=/d' -e '/ExecCondition=/d' -i $new_svcfile +cat >> "$new_svcfile" << ENDL +[Service] +ExecCondition=$XFS_HEALER_START_PROG --supported +ExecStart=$XFS_HEALER_START_PROG +ENDL +_systemd_reload + +# Emit the results of our editing to the full log. +systemctl cat "$new_healerstart_svc" >> $seqres.full + +echo "Start healer for everything" +_systemd_unit_start "$new_healerstart_svc" +find_healer_trace "$SCRATCH_MNT" + +echo "Restart healer for scratch FS" +_scratch_cycle_mount +find_healer_trace "$SCRATCH_MNT" + +echo "Healer testing done" | tee -a $seqres.full + +# success, all done +status=0 +exittests/xfs/1903.out new file +6
-0@@ -0,0 +1,6 @@ +QA output created by 1903 +Format and populate +Start healer on scratch FS +Start healer for everything +Restart healer for scratch FS +Healer testing done
[PATCH 15/14] xfs: test xfs_healer can follow private mntns mount moves
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure that when xfs_healer needs to reopen a filesystem to repair it, it can still find the filesystem even if it has been mount --move'd. This requires a bunch of private namespace magic.
--- tests/xfs/1904 | 129 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1904.out | 3 + 2 files changed, 132 insertions(+) create mode 100755 tests/xfs/1904 create mode 100755 tests/xfs/1904.out
tests/xfs/1904 new file +129 -0
@@ -0,0 +1,129 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Oracle. All Rights Reserved. +# +# FS QA Test 1904 +# +# Ensure that autonomous self healing fixes the filesystem correctly even if +# the original mount has moved somewhere else via --move. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +if [ -n "$IN_MOUNTNS" ]; then + _mount --make-rprivate / + findmnt -o TARGET,PROPAGATION >> $seqres.full + + _scratch_mount + _scratch_invoke_xfs_healer "$tmp.healer" --repair + + # Move the scratch filesystem to a completely different mountpoint so that + # we can test if the healer can find it again. + new_dir=$TEST_DIR/moocow + mkdir -p $new_dir + _mount --move $SCRATCH_MNT $new_dir + + df -t xfs >> $seqres.full + + # Access the broken directory to trigger a repair, then poll the directory + # for 5 seconds to see if it gets fixed without us needing to intervene. + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + _filter_scratch < $tmp.err | _filter_test_dir + try=0 + while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) + done + echo "try $try no longer saw corruption or gave up" >> $seqres.full + _filter_scratch < $tmp.err | _filter_test_dir + + # List the dirents of /victimdir to see if it stops reporting corruption + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=0 + while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) + done + echo "retry $try no longer saw corruption or gave up" >> $seqres.full + + new_dir_unmount() { + _unmount $new_dir + } + + # Unmount to kill the healer + _scratch_kill_xfs_healer new_dir_unmount + cat $tmp.healer >> $seqres.full + + # No need to clean up, the mount ns destructor will detach the + # filesystems for us. + exit +fi + +_cleanup() +{ + command -v _kill_fsstress &>/dev/null && _kill_fsstress + cd / + rm -r -f $tmp.* + if [ -n "$new_dir" ]; then + _unmount "$new_dir" &>/dev/null + rm -rf "$new_dir" + fi +} + +_require_unshare +_require_test +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full + +# mount --move only works if mount propagation is disabled, so we have to start +# a subshell with a separate mount namespace, disable propagation for the +# entire directory tree, and only then can we run our tests. +IN_MOUNTNS=1 unshare -m bash "$0" + +status=0 +exit
tests/xfs/1904.out new file +3 -0
@@ -0,0 +1,3 @@ +QA output created by 1904 +QA output created by 1904 +ls: reading directory 'TEST_DIR/moocow/some/victimdir': Structure needs cleaning
Zorro Lang
Reviewed-byRe: [PATCH 15/14] xfs: test xfs_healer can follow private mntns mount moves
Quoted 28 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that when
xfs_healerneeds to reopen a filesystem to repair it, it can still find the filesystem even if it has been mount --move'd. This requires a bunch of private namespace magic.Signed-off-by "Darrick J. Wong" <[email protected]>--- tests/xfs/1904 | 129 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1904.out | 3 + 2 files changed, 132 insertions(+) create mode 100755 tests/xfs/1904 create mode 100755 tests/xfs/1904.outtests/xfs/1904 new file +8
-0@@ -0,0 +1,129 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Oracle. All Rights Reserved. +# +# FS QA Test 1904 +# +# Ensure that autonomous self healing fixes the filesystem correctly even if +# the original mount has moved somewhere else via --move.
tests/xfs/1904 line 18
Quoted 10 lines
tests/xfs/1904 new file +10
-0+# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy +. ./common/systemd + +if [ -n "$IN_MOUNTNS" ]; then + _mount --make-rprivate /
I'd like to add this case and other cases related with mount propagation in this patchset to "mount" group. I'll do it when I merge this patchset. Others look and test good to me,
Quoted 121 lines
tests/xfs/1904 new file +111
-0+ findmnt -o TARGET,PROPAGATION >> $seqres.full + + _scratch_mount + _scratch_invoke_xfs_healer "$tmp.healer" --repair + + # Move the scratch filesystem to a completely different mountpoint so that + # we can test if the healer can find it again. + new_dir=$TEST_DIR/moocow + mkdir -p $new_dir + _mount --move $SCRATCH_MNT $new_dir + + df -t xfs >> $seqres.full + + # Access the broken directory to trigger a repair, then poll the directory + # for 5 seconds to see if it gets fixed without us needing to intervene. + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + _filter_scratch < $tmp.err | _filter_test_dir + try=0 + while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) + done + echo "try $try no longer saw corruption or gave up" >> $seqres.full + _filter_scratch < $tmp.err | _filter_test_dir + + # List the dirents of /victimdir to see if it stops reporting corruption + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=0 + while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) + done + echo "retry $try no longer saw corruption or gave up" >> $seqres.full + + new_dir_unmount() { + _unmount $new_dir + } + + # Unmount to kill the healer + _scratch_kill_xfs_healer new_dir_unmount + cat $tmp.healer >> $seqres.full + + # No need to clean up, the mount ns destructor will detach the + # filesystems for us. + exit +fi + +_cleanup() +{ + command -v _kill_fsstress &>/dev/null && _kill_fsstress + cd / + rm -r -f $tmp.* + if [ -n "$new_dir" ]; then + _unmount "$new_dir" &>/dev/null + rm -rf "$new_dir" + fi +} + +_require_unshare +_require_test +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full + +# mount --move only works if mount propagation is disabled, so we have to start +# a subshell with a separate mount namespace, disable propagation for the +# entire directory tree, and only then can we run our tests. +IN_MOUNTNS=1 unshare -m bash "$0" + +status=0 +exittests/xfs/1904.out new file +3
-0@@ -0,0 +1,3 @@ +QA output created by 1904 +QA output created by 1904 +ls: reading directory 'TEST_DIR/moocow/some/victimdir': Structure needs cleaning
Re: [PATCH 15/14] xfs: test xfs_healer can follow private mntns mount moves
Quoted 35 lines · Zorro Lang
Quoted 34 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Make sure that when
xfs_healerneeds to reopen a filesystem to repair it, it can still find the filesystem even if it has been mount --move'd. This requires a bunch of private namespace magic.Signed-off-by "Darrick J. Wong" <[email protected]>--- tests/xfs/1904 | 129 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1904.out | 3 + 2 files changed, 132 insertions(+) create mode 100755 tests/xfs/1904 create mode 100755 tests/xfs/1904.outtests/xfs/1904 new file +14
-0@@ -0,0 +1,129 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2026 Oracle. All Rights Reserved. +# +# FS QA Test 1904 +# +# Ensure that autonomous self healing fixes the filesystem correctly even if +# the original mount has moved somewhere else via --move. +# +. ./common/preamble +_begin_fstest auto selfhealing + +. ./common/filter +. ./common/fuzzy
Quoted 8 lines
tests/xfs/1904 line 18
Quoted 4 lines
tests/xfs/1904 new file +4
-0+. ./common/systemd + +if [ -n "$IN_MOUNTNS" ]; then + _mount --make-rprivate /I'd like to add this case and other cases related with mount propagation in this patchset to "mount" group. I'll do it when I merge this patchset. Others look and test good to me,
<nod> That sounds reasonable. Thanks for all the other minor touch-ups that you applied before merging into patches-in-queue!
--D
Quoted 126 lines
Reviewed-by Zorro Lang <[email protected]>Quoted 121 lines
tests/xfs/1904 new file +111
-0+ findmnt -o TARGET,PROPAGATION >> $seqres.full + + _scratch_mount + _scratch_invoke_xfs_healer "$tmp.healer" --repair + + # Move the scratch filesystem to a completely different mountpoint so that + # we can test if the healer can find it again. + new_dir=$TEST_DIR/moocow + mkdir -p $new_dir + _mount --move $SCRATCH_MNT $new_dir + + df -t xfs >> $seqres.full + + # Access the broken directory to trigger a repair, then poll the directory + # for 5 seconds to see if it gets fixed without us needing to intervene. + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + _filter_scratch < $tmp.err | _filter_test_dir + try=0 + while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "try $try saw corruption" >> $seqres.full + sleep 0.1 + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) + done + echo "try $try no longer saw corruption or gave up" >> $seqres.full + _filter_scratch < $tmp.err | _filter_test_dir + + # List the dirents of /victimdir to see if it stops reporting corruption + ls $new_dir/some/victimdir > /dev/null 2> $tmp.err + try=0 + while [ $try -lt 50 ] && grep -q 'Structure needs cleaning' $tmp.err; do + echo "retry $try still saw corruption" >> $seqres.full + sleep 0.1 + ls $SCRATCH_MNT/some/victimdir > /dev/null 2> $tmp.err + try=$((try + 1)) + done + echo "retry $try no longer saw corruption or gave up" >> $seqres.full + + new_dir_unmount() { + _unmount $new_dir + } + + # Unmount to kill the healer + _scratch_kill_xfs_healer new_dir_unmount + cat $tmp.healer >> $seqres.full + + # No need to clean up, the mount ns destructor will detach the + # filesystems for us. + exit +fi + +_cleanup() +{ + command -v _kill_fsstress &>/dev/null && _kill_fsstress + cd / + rm -r -f $tmp.* + if [ -n "$new_dir" ]; then + _unmount "$new_dir" &>/dev/null + rm -rf "$new_dir" + fi +} + +_require_unshare +_require_test +_require_scrub +_require_xfs_io_command "repair" # online repair support +_require_xfs_db_command "blocktrash" +_require_command "$XFS_HEALER_PROG" "xfs_healer" +_require_command "$XFS_PROPERTY_PROG" "xfs_property" +_require_scratch + +_scratch_mkfs >> $seqres.full +_scratch_mount + +_xfs_has_feature $SCRATCH_MNT rmapbt || \ + _notrun "reverse mapping required to test directory auto-repair" +_xfs_has_feature $SCRATCH_MNT parent || \ + _notrun "parent pointers required to test directory auto-repair" +_require_xfs_healer $SCRATCH_MNT --repair + +# Configure the filesystem for automatic repair of the filesystem. +$XFS_PROPERTY_PROG $SCRATCH_MNT set autofsck=repair >> $seqres.full + +# Create a largeish directory +dblksz=$(_xfs_get_dir_blocksize "$SCRATCH_MNT") +echo testdata > $SCRATCH_MNT/a +mkdir -p "$SCRATCH_MNT/some/victimdir" +for ((i = 0; i < (dblksz / 255); i++)); do + fname="$(printf "%0255d" "$i")" + ln $SCRATCH_MNT/a $SCRATCH_MNT/some/victimdir/$fname +done + +# Did we get at least two dir blocks? +dirsize=$(stat -c '%s' $SCRATCH_MNT/some/victimdir) +test "$dirsize" -gt "$dblksz" || echo "failed to create two-block directory" + +# Break the directory, remount filesystem +_scratch_unmount +_scratch_xfs_db -x \ + -c 'path /some/victimdir' \ + -c 'bmap' \ + -c 'dblock 1' \ + -c 'blocktrash -z -0 -o 0 -x 2048 -y 2048 -n 2048' >> $seqres.full + +# mount --move only works if mount propagation is disabled, so we have to start +# a subshell with a separate mount namespace, disable propagation for the +# entire directory tree, and only then can we run our tests. +IN_MOUNTNS=1 unshare -m bash "$0" + +status=0 +exittests/xfs/1904.out new file +3
-0@@ -0,0 +1,3 @@ +QA output created by 1904 +QA output created by 1904 +ls: reading directory 'TEST_DIR/moocow/some/victimdir': Structure needs cleaning