xfs_scrub_all.timer: don't run if /var/lib/xfsprogs is readonly
PATCH 1/1 xfs_scrub_all.timer: don't run if /var/lib/xfsprogs is readonly
From: Darrick J. Wong <[email protected]>
The xfs_scrub_all program wants to write a state file into the package state dir to keep track of how recently it performed a media scan. Don't allow the systemd timer to run if that path isn't writable.
---
scrub/Makefile | 6 +++++-
scrub/xfs_scrub_all.timer.in | 3 ++-
2 files changed, 7 insertions(+), 2 deletions(-)
rename scrub/{xfs_scrub_all.timer => xfs_scrub_all.timer.in} (77%)scrub/Makefile +5 -1
@@ -108,10 +108,14 @@ endif # Automatically trigger a media scan once per month XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_INTERVAL=1mo -LDIRT = $(XFS_SCRUB_ALL_PROG) $(XFS_SCRUB_FAIL_PROG) *.service *.cron +LDIRT = $(XFS_SCRUB_ALL_PROG) $(XFS_SCRUB_FAIL_PROG) *.service *.cron xfs_scrub_all.timer default: depend $(LTCOMMAND) $(XFS_SCRUB_ALL_PROG) $(XFS_SCRUB_FAIL_PROG) $(OPTIONAL_TARGETS) +xfs_scrub_all.timer: xfs_scrub_all.timer.in $(builddefs) + @echo " [SED] $@" + $(Q)$(SED) -e "s|@pkg_state_dir@|$(PKG_STATE_DIR)|g" < $< > $@ + xfs_scrub_all: xfs_scrub_all.in $(builddefs) @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \
scrub/xfs_scrub_all.timer.in renamed from scrub/xfs_scrub_all.timer +2 -1
@@ -1,10 +1,11 @@ # SPDX-License-Identifier: GPL-2.0-or-later # -# Copyright (C) 2018-2024 Oracle. All Rights Reserved. +# Copyright (C) 2018-2025 Oracle. All Rights Reserved. # Author: Darrick J. Wong <[email protected]> [Unit] Description=Periodic XFS Online Metadata Check for All Filesystems +ConditionPathIsReadWrite=@pkg_state_dir@ [Timer] # Run on Sunday at 3:10am, to avoid running afoul of DST changes
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The
xfs_scrub_allprogram wants to write a state file into the package state dir to keep track of how recently it performed a media scan. Don't allow the systemd timer to run if that path isn't writable.
Why would the path not be writable? Do we need a different place for it that is guaranteed to be writable even for setups they try to keep much of the system read-only?
Quoted 10 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The
xfs_scrub_allprogram wants to write a state file into the package state dir to keep track of how recently it performed a media scan. Don't allow the systemd timer to run if that path isn't writable.Why would the path not be writable? Do we need a different place for it that is guaranteed to be writable even for setups they try to keep much of the system read-only?
Eh, it's mostly to shut up systems where even /var/lib is readonly. IIRC systemd's volatile mode isn't quite that silly, but it'd be nice to avoid xfs_scrub_all repeatedly failing every time the timer fires.
OTOH maybe a better solution is just to run scrub in media scan mode if the media scan stamp file can't be opened for writing?
--D
Quoted 1 line · Darrick J. Wong
Eh, it's mostly to shut up systems where even
/var/libis readonly.
What are those systems? They must still provide some place writable for semi-persistant data, so we should look for that?
Quoted 5 lines
IIRC systemd's volatile mode isn't quite that silly, but it'd be nice to avoid
xfs_scrub_allrepeatedly failing every time the timer fires.OTOH maybe a better solution is just to run scrub in media scan mode if the media scan stamp file can't be opened for writing?
Or not run it at all? Either way I'd like to understand what causes this.
Quoted 5 lines · Christoph Hellwig
Quoted 1 line · Darrick J. Wong
Eh, it's mostly to shut up systems where even
/var/libis readonly.What are those systems? They must still provide some place writable for semi-persistant data, so we should look for that?
The particular place that I noticed this was on my fstests fleet, where the root filesystem is an ro nfs4 export. I forgot to configure an overlayfs for /var/lib/xfsprogs, so when I upgraded it to xfsprogs 6.12 and left the VMs running on a Sunday morning, they tried to start xfs_scrub_all and failed. Then the monitoring systems emailed me about that, and I got 150 emails. :(
This /should/ be a pretty uncommon situation since (AFAICT) most readonly-root systems set up a writable (and possibly volatile) /var/lib, but I thought I should just turn off the timer if it's going to fail anyway.
Quoted 8 lines
Quoted 5 lines
IIRC systemd's volatile mode isn't quite that silly, but it'd be nice to avoid
xfs_scrub_allrepeatedly failing every time the timer fires.OTOH maybe a better solution is just to run scrub in media scan mode if the media scan stamp file can't be opened for writing?
Or not run it at all? Either way I'd like to understand what causes this.
<nod>
--D
Christoph Hellwig
Reviewed-byQuoted 11 lines · Darrick J. Wong
The particular place that I noticed this was on my fstests fleet, where the root filesystem is an ro nfs4 export. I forgot to configure an overlayfs for
/var/lib/xfsprogs, so when I upgraded it to xfsprogs 6.12 and left the VMs running on a Sunday morning, they tried to startxfs_scrub_alland failed. Then the monitoring systems emailed me about that, and I got 150 emails. :(This /should/ be a pretty uncommon situation since (AFAICT) most readonly-root systems set up a writable (and possibly volatile)
/var/lib, but I thought I should just turn off the timer if it's going to fail anyway.
Can you write this up in a comment in the systemd unit file?
With that the patch looks fine to me:
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
The particular place that I noticed this was on my fstests fleet, where the root filesystem is an ro nfs4 export. I forgot to configure an overlayfs for
/var/lib/xfsprogs, so when I upgraded it to xfsprogs 6.12 and left the VMs running on a Sunday morning, they tried to startxfs_scrub_alland failed. Then the monitoring systems emailed me about that, and I got 150 emails. :(
Quoted 6 lines
Quoted 4 lines
This /should/ be a pretty uncommon situation since (AFAICT) most readonly-root systems set up a writable (and possibly volatile)
/var/lib, but I thought I should just turn off the timer if it's going to fail anyway.Can you write this up in a comment in the systemd unit file?
Will do!
Quoted 3 lines
With that the patch looks fine to me:
Reviewed-by Christoph Hellwig <[email protected]>
Thank you.
--D
PATCH v2 1/1 xfs_scrub_all.timer: don't run if /var/lib/xfsprogs is readonly
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The xfs_scrub_all program wants to write a state file into the package state dir to keep track of how recently it performed a media scan. Don't allow the systemd timer to run if that path isn't writable.
---
v2: add some comments to timer file
--- scrub/Makefile | 6 +++++- scrub/xfs_scrub_all.timer | 16 ---------------- scrub/xfs_scrub_all.timer.in | 23 +++++++++++++++++++++++ 3 files changed, 28 insertions(+), 17 deletions(-) delete mode 100644 scrub/xfs_scrub_all.timer create mode 100644 scrub/xfs_scrub_all.timer.in
scrub/Makefile +5 -1
@@ -108,10 +108,14 @@ endif # Automatically trigger a media scan once per month XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_INTERVAL=1mo -LDIRT = $(XFS_SCRUB_ALL_PROG) $(XFS_SCRUB_FAIL_PROG) *.service *.cron +LDIRT = $(XFS_SCRUB_ALL_PROG) $(XFS_SCRUB_FAIL_PROG) *.service *.cron xfs_scrub_all.timer default: depend $(LTCOMMAND) $(XFS_SCRUB_ALL_PROG) $(XFS_SCRUB_FAIL_PROG) $(OPTIONAL_TARGETS) +xfs_scrub_all.timer: xfs_scrub_all.timer.in $(builddefs) + @echo " [SED] $@" + $(Q)$(SED) -e "s|@pkg_state_dir@|$(PKG_STATE_DIR)|g" < $< > $@ + xfs_scrub_all: xfs_scrub_all.in $(builddefs) @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \
scrub/xfs_scrub_all.timer deleted +0 -16
@@ -1,16 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-or-later -# -# Copyright (C) 2018-2024 Oracle. All Rights Reserved. -# Author: Darrick J. Wong <[email protected]> - -[Unit] -Description=Periodic XFS Online Metadata Check for All Filesystems - -[Timer] -# Run on Sunday at 3:10am, to avoid running afoul of DST changes -OnCalendar=Sun *-*-* 03:10:00 -RandomizedDelaySec=60 -Persistent=true - -[Install] -WantedBy=timers.target
scrub/xfs_scrub_all.timer.in new file +23 -0
@@ -0,0 +1,23 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2018-2025 Oracle. All Rights Reserved. +# Author: Darrick J. Wong <[email protected]> + +[Unit] +Description=Periodic XFS Online Metadata Check for All Filesystems + +# The xfs_scrub_all program records the last time that it performed a media +# scan in @pkg_state_dir@. If this path is not writable, the program +# aborts and systemd records this as a failure. Disable the timer if the path +# is not writable. This should be an uncommon situation since most +# readonly-root systems set that up to be writable (and possibly volatile). +ConditionPathIsReadWrite=@pkg_state_dir@ + +[Timer] +# Run on Sunday at 3:10am, to avoid running afoul of DST changes +OnCalendar=Sun *-*-* 03:10:00 +RandomizedDelaySec=60 +Persistent=true + +[Install] +WantedBy=timers.target