xfs: resolve close() deadlocks on frozen filesystems
v4 cover xfs: resolve close() deadlocks on frozen filesystems
From: Aditya Prakash Srivastava <[email protected]>
Hi Carlos and Christoph,
Apologies for the mailing list noise in v3; those patches were accidentally sent as unthreaded emails due to a local Git configuration. This v4 series is properly threaded under the cover letter standalone, and has a distinct subject line to prevent mail client threading collisions.
This is version 4 of the patch series addressing the close() system call hanging indefinitely on frozen XFS filesystems (Bugzilla #205833).
Based on Christoph's feedback, I have made the following improvements:
- Split the changes into a clean, 2-patch series separating the new trylock flag implementation from the deadlock fix itself.
- Extracted the case history, reproducer code, and discussion from the commit logs into this cover letter to keep commit logs surgically focused.
- Adjusted the parameter name in xfs_free_eofblocks() to trans_flags to make its usage stand out.
- Implemented a much cleaner, race-free state preservation logic in xfs_file_release() by omitting pre-setting XFS_EOFBLOCKS_RELEASED on the inode, setting it only upon successful truncation.
- Simplified the transaction allocation block, renamed the flag to XFS_TRANS_WRITECOUNT_TRYLOCK, and added the requested mutual-exclusivity assertion in __xfs_trans_alloc().
- Set up this series to be posted standalone (not threaded as a reply) per the standard kernel guidelines.
As requested, I have also submitted the corresponding regression test to the xfstests mailing list (using tests/xfs/842 with a GPLv2-licensed helper program). The fstests maintainer (Zorro Lang) reviewed the test and has indicated that he will wait for this kernel patch to make it through before merging the test suite additions.
THE REAL-WORLD IMPACT (BUGZILLA & DOWNSTREAM CASES)
==================================================
When speculative post-EOF blocks are closed on XFS, the release path synchronously attempts to free them via xfs_free_eofblocks(). This allocates a write transaction (xfs_trans_alloc) which blocks indefinitely on the superblock freeze write lock (sb_start_intwrite) under fsfreeze.
This behavior has a long history of causing severe system disruption:
- Downstream Red Hat Bugzilla 1474726 (dating back to 2017) details complete system hangs during system backups when rsync and fsfreeze are used. Even seemingly harmless read-only commands like 'cat /var/log/messages' would hang on close() in __sb_start_write via xfs_free_eofblocks, requiring a hard reboot.
- Downstream LeApp integration test scenarios consistently hit this hang.
Hanging on close() frequently triggers container healthcheck failures, systemd service timeouts, and cluster failover cascades, which is disruptive to user-space applications that view close() as resource reclamation. No other major Linux filesystem (ext4, btrfs, etc.) synchronously allocates write transactions during close() system calls.
THE SOLUTION: NON-BLOCKING SUPERBLOCK TRYLOCK
=============================================
Instead of performing racy pre-checks, this series introduces XFS_TRANS_WRITECOUNT_TRYLOCK. When specified, __xfs_trans_alloc() attempts to obtain freeze protection using sb_start_intwrite_trylock(). If that fails, it aborts allocation gracefully and returns -EAGAIN.
We then pass XFS_TRANS_WRITECOUNT_TRYLOCK during xfs_file_release(). If the truncation fails due to a frozen filesystem (-EAGAIN), we cleanly bypass setting XFS_EOFBLOCKS_RELEASED on the inode, ensuring subsequent releases or the background blockgc garbage collector can successfully clean them up once thawed.
REPRODUCER DETAILS (GPLV2 LICENSED)
===================================
As requested, I have added a GPLv2-compatible license to the C reproducer provided below, and I have also sent a corresponding patch to the xfstests mailing list. The fstests maintainer (Zorro Lang) reviewed the patch and indicated that he will wait for this kernel patch series to be merged before pulling the test suite additions.
Compile with -pthread:
/*
* GPLv2-compatible XFS freeze close() hang reproducer.
* Copyright (c) 2026 Aditya Prakash Srivastava. All Rights Reserved.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <pthread.h>
#include <sys/ioctl.h>
#include <sys/vfs.h>
#include <linux/fs.h>
#include <libgen.h>
volatile int close_started = 0;
volatile int close_completed = 0;
void *close_thread(void *arg) {
int fd = *(int *)arg;
close_started = 1;
close(fd);
close_completed = 1;
return NULL;
}
int main(int argc, char *argv[]) {
struct statfs sfs;
if (statfs(argv[1], &sfs) < 0) {
char *dir_buf = strdup(argv[1]);
char *parent_dir = dirname(dir_buf);
if (statfs(parent_dir, &sfs) < 0) {
perror("statfs");
free(dir_buf);
return 1;
}
free(dir_buf);
}
if (sfs.f_type != 0x58465342) return 1;
int freeze_fd = open(dirname(strdup(argv[1])), O_RDONLY);
int write_fd = open(argv[1], O_WRONLY | O_CREAT | O_TRUNC, 0644);
char buf[65536] = {0};
for (int i = 0; i < 320; i++) write(write_fd, buf, sizeof(buf));
ioctl(freeze_fd, FIFREEZE, 0);
pthread_t thread;
pthread_create(&thread, NULL, close_thread, &write_fd);
while (!close_started) usleep(1000);
usleep(1000000); // Wait 1s
if (!close_completed) printf("SUCCESS: close() hung!\n");
ioctl(freeze_fd, FITHAW, 0);
pthread_join(thread, NULL);
unlink(argv[1]);
return 0;
}Aditya Prakash Srivastava (2):
xfs: add a XFS_TRANS_WRITECOUNT_TRYLOCK flag
xfs: prevent close() from hanging on frozen filesystems
fs/xfs/libxfs/xfs_shared.h | 3 +++ fs/xfs/xfs_bmap_util.c | 9 +++++---- fs/xfs/xfs_bmap_util.h | 2 +- fs/xfs/xfs_file.c | 8 +++++--- fs/xfs/xfs_icache.c | 2 +- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_trans.c | 12 +++++++++++- 7 files changed, 27 insertions(+), 11 deletions(-)
Signature · 1 line
2.47.3
PATCH v4 1/2 xfs: add a XFS_TRANS_WRITECOUNT_TRYLOCK flag
From: Aditya Prakash Srivastava <[email protected]>
Introduce a new transaction allocation flag, XFS_TRANS_WRITECOUNT_TRYLOCK. When this flag is specified, __xfs_trans_alloc() attempts to obtain freeze protection using sb_start_intwrite_trylock() instead of blocking indefinitely on sb_start_intwrite().
If the trylock fails, the allocation is aborted gracefully: the freshly allocated transaction handle is freed, and the function returns the appropriate error pointer ERR_PTR(-EAGAIN), which is then propagated to the caller by xfs_trans_alloc().
Also add an assertion in __xfs_trans_alloc() to ensure that both XFS_TRANS_NO_WRITECOUNT and XFS_TRANS_WRITECOUNT_TRYLOCK are never specified at the same time, as they are mutually exclusive.
--- fs/xfs/libxfs/xfs_shared.h | 3 +++ fs/xfs/xfs_trans.c | 12 +++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_shared.h +3 -0
@@ -164,6 +164,9 @@ void xfs_log_get_max_trans_res(struct xfs_mount *mp,/* Transaction has locked the rtbitmap and rtsum inodes */ #define XFS_TRANS_RTBITMAP_LOCKED (1u << 9) +/* Try lock filesystem superblock for freeze protection */ +#define XFS_TRANS_WRITECOUNT_TRYLOCK (1u << 10) +/* * Field values for xfs_trans_mod_sb. */
fs/xfs/xfs_trans.c +11 -1
@@ -216,10 +216,18 @@ __xfs_trans_alloc(struct xfs_trans *tp;ASSERT (!(flags &XFS_TRANS_RES_FDBLKS ) ||xfs_has_lazysbcount (mp)); +ASSERT (!((flags &XFS_TRANS_NO_WRITECOUNT ) && + (flags &XFS_TRANS_WRITECOUNT_TRYLOCK ))); tp =kmem_cache_zalloc (xfs_trans_cache,GFP_KERNEL | __GFP_NOFAIL); -if (!(flags &XFS_TRANS_NO_WRITECOUNT )) +if (flags &XFS_TRANS_WRITECOUNT_TRYLOCK ) { +if (!sb_start_intwrite_trylock (mp->m_super)) { +kmem_cache_free (xfs_trans_cache, tp); +return ERR_PTR (-EAGAIN ); + } + }else if (!(flags &XFS_TRANS_NO_WRITECOUNT )) {sb_start_intwrite (mp->m_super); + }xfs_trans_set_context (tp); tp->t_flags = flags; tp->t_mountp = mp;
@@ -252,6 +260,8 @@ xfs_trans_alloc(*/ retry: tp =__xfs_trans_alloc (mp, flags); +if (IS_ERR (tp)) +return PTR_ERR (tp);WARN_ON (mp->m_super->s_writers.frozen ==SB_FREEZE_COMPLETE ); error =xfs_trans_reserve (tp, resp, blocks, rtextents);if (error == -ENOSPC && want_retry) {
Signature · 1 line
2.47.3
Re: [PATCH v4 1/2] xfs: add a XFS_TRANS_WRITECOUNT_TRYLOCK flag
Looks good:
PATCH v4 2/2 xfs: prevent close() from hanging on frozen filesystems
In reply to Aditya SrivastavaFrom: Aditya Prakash Srivastava <[email protected]>
When a file with active speculative post-EOF preallocations is closed, xfs_file_release() synchronously triggers xfs_free_eofblocks() to clean them up. This requires allocating a write transaction (xfs_trans_alloc), which blocks indefinitely if the filesystem is currently frozen or in the process of freezing, as it waits to acquire the superblock's write lock.
As a result, a close() system call on a read-write file descriptor can hang indefinitely in percpu_rwsem_wait() until the filesystem is thawed, even if the file is closed by a non-writer process or after all writing activity has already ceased.
To fix this properly and avoid any potential race conditions where a freeze might come in immediately after a writable check, pass the new XFS_TRANS_WRITECOUNT_TRYLOCK flag to xfs_trans_alloc() when freeing speculative preallocations in xfs_file_release().
If xfs_free_eofblocks() returns -EAGAIN on a trylock failure, we cleanly bypass setting XFS_EOFBLOCKS_RELEASED on the inode, ensuring subsequent releases or the background blockgc garbage collector can successfully retry the cleanup once the filesystem thaws.
Also, rename the flags parameter in xfs_free_eofblocks() to trans_flags as suggested to make its usage stand out, and update existing callers to pass 0 to preserve standard blocking paths.
--- fs/xfs/xfs_bmap_util.c | 9 +++++---- fs/xfs/xfs_bmap_util.h | 2 +- fs/xfs/xfs_file.c | 8 +++++--- fs/xfs/xfs_icache.c | 2 +- fs/xfs/xfs_inode.c | 2 +- 5 files changed, 13 insertions(+), 10 deletions(-)
fs/xfs/xfs_bmap_util.c +5 -4
@@ -574,7 +574,8 @@ xfs_can_free_eofblocks(*/ int xfs_free_eofblocks ( -struct xfs_inode *ip) +struct xfs_inode *ip, +uint trans_flags) {struct xfs_trans *tp;struct xfs_mount *mp = ip->i_mount;
@@ -604,9 +605,9 @@ xfs_free_eofblocks(return 0 ; } - error =xfs_trans_alloc (mp, &M_RES (mp)->tr_itruncate,0 ,0 ,0 , &tp); + error =xfs_trans_alloc (mp, &M_RES (mp)->tr_itruncate,0 ,0 , trans_flags, &tp);if (error) { -ASSERT (xfs_is_shutdown (mp)); +ASSERT (error == -EAGAIN ||xfs_is_shutdown (mp));return error; }
@@ -928,7 +929,7 @@ xfs_prepare_shift(* into the accessible region of the file. */ if (xfs_can_free_eofblocks (ip)) { - error =xfs_free_eofblocks (ip); + error =xfs_free_eofblocks (ip,0 );if (error)return error; }
fs/xfs/xfs_bmap_util.h +1 -1
@@ -66,7 +66,7 @@ int xfs_insert_file_space(struct xfs_inode *, xfs_off_t offset,/* EOF block manipulation functions */ bool xfs_can_free_eofblocks (struct xfs_inode *ip); -int xfs_free_eofblocks (struct xfs_inode *ip); +int xfs_free_eofblocks (struct xfs_inode *ip,uint trans_flags);int xfs_swap_extents (struct xfs_inode *ip,struct xfs_inode *tip,struct xfs_swapext *sx);
fs/xfs/xfs_file.c +5 -3
@@ -1806,9 +1806,11 @@ xfs_file_release(*/ if (!xfs_iflags_test (ip,XFS_EOFBLOCKS_RELEASED ) &&xfs_ilock_nowait (ip,XFS_IOLOCK_EXCL )) { -if (xfs_can_free_eofblocks (ip) && - !xfs_iflags_test_and_set (ip,XFS_EOFBLOCKS_RELEASED )) -xfs_free_eofblocks (ip); +if (!xfs_iflags_test (ip,XFS_EOFBLOCKS_RELEASED ) && +xfs_can_free_eofblocks (ip) && + !xfs_free_eofblocks (ip,XFS_TRANS_WRITECOUNT_TRYLOCK )) +xfs_iflags_set (ip,XFS_EOFBLOCKS_RELEASED ); +xfs_iunlock (ip,XFS_IOLOCK_EXCL ); }
fs/xfs/xfs_icache.c +1 -1
@@ -1259,7 +1259,7 @@ xfs_inode_free_eofblocks( *lockflags |=XFS_IOLOCK_EXCL ;if (xfs_can_free_eofblocks (ip)) -return xfs_free_eofblocks (ip); +return xfs_free_eofblocks (ip,0 );/* inode could be preallocated */ trace_xfs_inode_free_eofblocks_invalid (ip);
fs/xfs/xfs_inode.c +1 -1
@@ -1423,7 +1423,7 @@ xfs_inactive(* reference to the inode at this point anyways. */ if (xfs_can_free_eofblocks (ip)) - error =xfs_free_eofblocks (ip); + error =xfs_free_eofblocks (ip,0 ); goto out; }
Signature · 1 line
2.47.3
Re: [PATCH v4 2/2] xfs: prevent close() from hanging on frozen filesystems
Quoted 3 lines · Aditya Srivastava
Also, rename the flags parameter in
xfs_free_eofblocks()totrans_flagsas suggested to make its usage stand out, and update existing callers to pass 0 to preserve standard blocking paths.
It is only renamed compared to the previous patches, because this patch actually adds it.
fs/xfs/xfs_bmap_util.c xfs_free_eofblocks( · line 608
Quoted 2 lines
fs/xfs/xfs_bmap_util.c +1
-1- error =xfs_trans_alloc (mp, &M_RES (mp)->tr_itruncate,0 ,0 ,0 , &tp); + error =xfs_trans_alloc (mp, &M_RES (mp)->tr_itruncate,0 ,0 , trans_flags, &tp);
Overly long line here.
Otherwise this looks good.