Skip to contentSkip to conversation

New: XFS: oversized xattr transaction causes unkillable hang in xlog_grant_head_check

Bug 221915 [email protected] 1 participant 1 message Aug 24

lore
Share & tools
lore
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=221915

Bug ID: 221915
Summary: XFS: oversized xattr transaction causes unkillable
hang in xlog_grant_head_check
Product: File System
Version: 2.5
Hardware: All
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: XFS
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Created attachment 310718
--> https://bugzilla.kernel.org/attachment.cgi?id=310718&action=edit
submit

Linux mainline 7.2-rc7 still contains a reproducible XFS hang in the log grant reservation path.

While fuzzing current mainline with syzkaller, I reproduced an old class of hang previously reported by syzbot as:

```text
INFO: task hung in xlog_grant_head_check
```

The issue can be reduced to mounting a crafted XFS filesystem image with a small log and issuing a single large security.ima xattr operation.

The minimized syzkaller sequence is approximately:

```text
syz_mount_image$xfs(..., "./file0", ..., image)

setxattr$security_ima("./file0", "security.ima",
                     {sha1, ...}, 0xfffe, 0)
```

The task then becomes permanently blocked:

```text
INFO: task syz.0.17:10462 blocked for more than 143 seconds.

Call Trace:
 __schedule+0x12e4/0x64e0
 schedule+0xe7/0x2d0
 xlog_grant_head_wait+0x13f/0xd00 fs/xfs/xfs_log.c:217
 xlog_grant_head_check+0x48f/0x4e0 fs/xfs/xfs_log.c:279
 xfs_log_reserve+0x2d2/0x850 fs/xfs/xfs_log.c:387
 xfs_trans_reserve fs/xfs/trans.c:174 [inline]
 xfs_trans_alloc+0x3c5/0xd40 fs/xfs/xfs_trans.c:256
 xfs_trans_alloc_inode+0x198/0x3b0 fs/xfs/xfs_trans.c:1080
 xfs_attr_set+0x40b/0x10f0 fs/xfs/libxfs/xfs_attr.c:1184
 xfs_attr_change+0x379/0x500 fs/xfs/xfs_xattr.c:128
 ...
 __x64_sys_setxattr+0xc6/0x140 fs/xattr.c:764
```

## Trigger

The reproducer uses:

```text
- a crafted XFS v5 filesystem image
- a relatively small XFS log
- one approximately 64 KB security.ima xattr value
```

The large xattr causes XFS to request a transaction reservation that appears to be larger than the amount of log space that can ever become available on the crafted filesystem.

The task enters:

```text
xfs_log_reserve()
  -> xlog_grant_head_check()
     -> xlog_grant_head_wait()
```

and does not return.

The wait does not appear to resolve even after log tail progress, and the blocked task is effectively unkillable while waiting in D state.

Eventually the kernel reports the task through the hung-task detector. With hung_task_panic=1, this can also result in a kernel panic.

## Suspected cause

My current hypothesis is that XFS can enter xlog_grant_head_wait() for a transaction reservation that is impossible to satisfy.

For this crafted filesystem, the requested transaction reservation appears to exceed the usable size of the XFS log.

Normally, xlog_grant_head_wait() waits for log tail movement to free sufficient reservation space. However, if:

```text
requested reservation > maximum usable log space
```

then no amount of log tail pushing can ever make the reservation satisfiable.

The task therefore waits indefinitely.

This suggests that the reservation path may be missing a sanity check rejecting transactions whose requested reservation exceeds the maximum amount of log space that can ever be granted.

Another possible fix would be for the xattr path to split sufficiently large operations into smaller transactions.

This analysis has not yet been fully proven, but it is consistent with the minimized reproducer and the observed permanent wait.

## Expected behavior

A userspace operation that requires a transaction larger than the filesystem log can accommodate should fail cleanly, for example with an appropriate error such as -ENOSPC or another reservation failure.

It should not result in an indefinitely blocked and unkillable task.

## Actual behavior

A single large setxattr() can cause the task to block indefinitely in:

```text
xlog_grant_head_wait()
```

The operation never returns, and the kernel eventually reports:

```text
INFO: task ... blocked for more than 143 seconds
```

## Reproducer

The complete syzkaller reproducer consists of the crafted XFS image and the two-operation program described above.

```text
1. Mount crafted XFS image.
2. setxattr("security.ima", ..., size=0xfffe)
3. Wait for the task to become permanently blocked in
   xlog_grant_head_wait().
```

The full syzkaller reproducer, filesystem image, kernel configuration, and console log can be attached to this report.

## Environment

```text
Kernel:
7.2.0-rc7-00232-gdcb68831eac7

Tree:
stock mainline

Filesystem:
XFS v5

Configuration:
syzkaller upstream KASAN configuration

Architecture:
x86-64
```

## Previous report

A similar syzbot report existed in 2018 under the title:

```text
INFO: task hung in xlog_grant_head_check
```

The tracker has been dormant for several years and the previous cause analysis/bisection did not result in a confirmed fix.

The issue is still reproducible on current mainline 7.2-rc7 with a very small reproducer.

References:

https://lwn.net/Articles/755633/

https://lkml.org/lkml/2018/5/22/1056

Found-by: syzkaller on linux-7.2.0-rc7-00232-gdcb68831eac7

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.