New: XFS: oversized xattr transaction causes unkillable hang in xlog_grant_head_check
https://bugzilla.kernel.org/show_bug.cgi?id=221915
Bug ID: 221915
Summary: XFS: oversized xattr transaction causes unkillable
hang in xlog_grant_head_check
Product: File System
Version: 2.5
Hardware: All
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: XFS
Assignee: [email protected]
Reporter: [email protected]
Regression: No
Created attachment 310718
--> https://bugzilla.kernel.org/attachment.cgi?id=310718&action=edit
submit
Linux mainline 7.2-rc7 still contains a reproducible XFS hang in the log grant reservation path.
While fuzzing current mainline with syzkaller, I reproduced an old class of hang previously reported by syzbot as:
```text INFO: task hung in xlog_grant_head_check ```
The issue can be reduced to mounting a crafted XFS filesystem image with a small log and issuing a single large security.ima xattr operation.
The minimized syzkaller sequence is approximately:
```text
syz_mount_image$xfs(..., "./file0", ..., image)
setxattr$security_ima("./file0", "security.ima",
{sha1, ...}, 0xfffe, 0)
```The task then becomes permanently blocked:
```text INFO: task syz.0.17:10462 blocked for more than 143 seconds. Call Trace: __schedule+0x12e4/0x64e0 schedule+0xe7/0x2d0 xlog_grant_head_wait+0x13f/0xd00 fs/xfs/xfs_log.c:217 xlog_grant_head_check+0x48f/0x4e0 fs/xfs/xfs_log.c:279 xfs_log_reserve+0x2d2/0x850 fs/xfs/xfs_log.c:387 xfs_trans_reserve fs/xfs/trans.c:174 [inline] xfs_trans_alloc+0x3c5/0xd40 fs/xfs/xfs_trans.c:256 xfs_trans_alloc_inode+0x198/0x3b0 fs/xfs/xfs_trans.c:1080 xfs_attr_set+0x40b/0x10f0 fs/xfs/libxfs/xfs_attr.c:1184 xfs_attr_change+0x379/0x500 fs/xfs/xfs_xattr.c:128 ... __x64_sys_setxattr+0xc6/0x140 fs/xattr.c:764 ```
## Trigger
The reproducer uses:
```text - a crafted XFS v5 filesystem image - a relatively small XFS log - one approximately 64 KB security.ima xattr value ```
The large xattr causes XFS to request a transaction reservation that appears to be larger than the amount of log space that can ever become available on the crafted filesystem.
The task enters:
```text
xfs_log_reserve()
-> xlog_grant_head_check()
-> xlog_grant_head_wait()
```and does not return.
The wait does not appear to resolve even after log tail progress, and the blocked task is effectively unkillable while waiting in D state.
Eventually the kernel reports the task through the hung-task detector. With hung_task_panic=1, this can also result in a kernel panic.
## Suspected cause
My current hypothesis is that XFS can enter xlog_grant_head_wait() for a transaction reservation that is impossible to satisfy.
For this crafted filesystem, the requested transaction reservation appears to exceed the usable size of the XFS log.
Normally, xlog_grant_head_wait() waits for log tail movement to free sufficient reservation space. However, if:
```text requested reservation > maximum usable log space ```
then no amount of log tail pushing can ever make the reservation satisfiable.
The task therefore waits indefinitely.
This suggests that the reservation path may be missing a sanity check rejecting transactions whose requested reservation exceeds the maximum amount of log space that can ever be granted.
Another possible fix would be for the xattr path to split sufficiently large operations into smaller transactions.
This analysis has not yet been fully proven, but it is consistent with the minimized reproducer and the observed permanent wait.
## Expected behavior
A userspace operation that requires a transaction larger than the filesystem log can accommodate should fail cleanly, for example with an appropriate error such as -ENOSPC or another reservation failure.
It should not result in an indefinitely blocked and unkillable task.
## Actual behavior
A single large setxattr() can cause the task to block indefinitely in:
```text xlog_grant_head_wait() ```
The operation never returns, and the kernel eventually reports:
```text INFO: task ... blocked for more than 143 seconds ```
## Reproducer
The complete syzkaller reproducer consists of the crafted XFS image and the two-operation program described above.
```text
1. Mount crafted XFS image.
2. setxattr("security.ima", ..., size=0xfffe)
3. Wait for the task to become permanently blocked in
xlog_grant_head_wait().
```The full syzkaller reproducer, filesystem image, kernel configuration, and console log can be attached to this report.
## Environment
```text Kernel: 7.2.0-rc7-00232-gdcb68831eac7 Tree: stock mainline Filesystem: XFS v5 Configuration: syzkaller upstream KASAN configuration Architecture: x86-64 ```
## Previous report
A similar syzbot report existed in 2018 under the title:
```text INFO: task hung in xlog_grant_head_check ```
The tracker has been dormant for several years and the previous cause analysis/bisection did not result in a confirmed fix.
The issue is still reproducible on current mainline 7.2-rc7 with a very small reproducer.
References:
https://lwn.net/Articles/755633/
https://lkml.org/lkml/2018/5/22/1056
Found-by: syzkaller on linux-7.2.0-rc7-00232-gdcb68831eac7
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.