Strange behavior with scrub, quotas, and snapshots
Hello.
I am struggling with a poorly behaved BTRFS volume.
It is a simple volume, consuming only one partition, with no redundancy for data. The underlying media, which is NVME, appears to be completely healthy, according to SMART.
Several weeks ago, the volume became unwriteable, with false reports generated that no space was available, whenever would be attempted the creation of a new file.
After a chaotic intermix of balance and scrub operations, as well as the deletion of many snapshots and a few large files, a message eventually appeared in the kernel log reporting the discovery of corruption in the space cache, that had since been resolved.
After, the volume again was usable, with no problems writing files.
Incidentally, I later upgraded to space cache version 2.
Although the episode of complete dysfunction has ended, problems remain. In particular are two observations.
First, scrub operations complete almost immediately, reporting a status of "finished", with no errors found.
However, as seen also seen in the following console capture, the reported amount of total data scanned is only a fraction of the total used space:
---
$ btrfs --version btrfs-progs v6.6.3
$ btrfs fi show Label: none uuid: bbac86e5-eaba-45bf-bbaa-c2494e11831a Total devices 1 FS bytes used 671.85GiB devid 1 size 831.26GiB used 831.26GiB path /dev/nvme0n1p5
$ sudo btrfs filesystem df / Data, single: total=813.13GiB, used=662.49GiB System, DUP: total=32.00MiB, used=144.00KiB Metadata, DUP: total=9.03GiB, used=6.26GiB GlobalReserve, single: total=512.00MiB, used=0.00B
$ sudo btrfs scrub start -B / Starting scrub on devid 1 scrub done for bbac86e5-eaba-45bf-bbaa-c2494e11831a Scrub started: Sun Apr 26 19:24:02 2026 Status: finished Duration: 0:00:11 Total to scrub: 23.37GiB Rate: 2.12GiB/s Error summary: no errors found ---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
Thank you.
[21:00:06] Removing snapshot: 2026-04-23_20-00-02
[21:00:06] Deleting subvolume: @ (Id:60338)
[21:00:06] btrfs subvolume delete --commit-after '/run/timeshift/105175/backup/timeshift-btrfs/snapshots/2026-04-23_20-00-02/@' [21:00:06] Waiting on btrfs to finish deleting...
[21:04:55] Deleted subvolume: @ (Id:60338)
[21:04:55] Rescanning quotas...
[21:04:55] Destroying qgroup: 0/60338
[21:04:55] btrfs qgroup destroy 0/60338 '/run/timeshift/105175/backup'
[21:04:55] E: Failed to destroy qgroup: '0/60338'
[21:04:55] E: Failed to remove snapshot: 2026-04-23_20-00-02
[21:04:55] ------------------------------------------------------------------------------
[21:04:55] ------------------------------------------------------------------------------
[21:04:55] Removing snapshot: 2026-04-23_21-58-52
[21:04:55] Deleting subvolume: @ (Id:60350)
[21:04:55] btrfs subvolume delete --commit-after '/run/timeshift/105175/backup/timeshift-btrfs/snapshots/2026-04-23_21-58-52/@' [21:04:55] Waiting on btrfs to finish deleting...
[21:08:45] Deleted subvolume: @ (Id:60350)
[21:08:45] Rescanning quotas...
[21:08:45] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:46] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:47] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:48] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:50] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:51] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:52] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:53] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:54] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:55] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:56] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:57] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:58] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:08:59] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:00] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:01] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:02] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:03] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:04] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:05] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:06] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:07] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:08] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:09] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:10] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:11] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:12] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:13] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:14] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:15] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:16] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:17] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:18] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:19] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:20] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:21] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:22] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:23] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:24] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:25] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:26] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:27] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:28] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:29] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:30] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:31] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:32] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:33] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:34] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:35] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:36] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:37] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:38] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:39] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:40] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:41] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:42] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:43] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:44] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:45] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:46] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:47] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:48] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:49] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:50] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:51] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:52] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:53] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:54] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:55] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:56] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:57] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:58] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:09:59] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:00] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:01] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:02] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:03] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:04] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:05] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:06] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:07] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:08] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:09] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:10] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:11] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:12] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:13] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:14] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:15] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:16] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:17] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:18] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:19] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:20] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:21] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:22] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:23] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:24] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:25] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:26] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:27] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:28] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:29] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:30] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:31] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:32] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:33] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:34] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:35] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:36] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:37] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:38] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:39] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:40] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:41] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:42] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:43] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:44] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:45] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:47] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:48] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:49] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:50] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:51] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:52] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:53] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:54] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:55] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:56] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:57] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:58] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:10:59] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:00] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:01] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:02] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:03] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:04] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:05] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:06] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:07] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:08] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:09] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:10] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:11] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:12] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:13] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:14] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:15] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:16] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:17] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:18] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:19] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:20] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:21] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:22] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:23] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:24] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:25] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:26] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:27] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:28] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:29] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:30] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:31] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:32] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:33] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:34] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:35] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:36] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:37] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:38] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:39] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:40] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:41] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:42] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:43] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:44] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:45] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:46] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:47] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:48] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:49] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:50] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:51] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:52] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:53] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:54] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:55] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:56] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:57] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:58] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:11:59] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:00] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:01] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:02] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:03] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:04] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:05] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:06] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:07] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:08] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:09] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:10] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:11] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:12] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:13] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:14] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:15] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:16] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:17] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:18] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:19] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:20] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:21] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:22] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:23] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:24] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:25] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:26] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:27] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:28] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:29] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:30] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:31] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:32] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:33] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:34] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:35] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:36] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:37] Still rescanning quotas... ERROR: quota rescan failed: Operation now in progress
[21:12:38] Destroying qgroup: 0/60350
[21:12:38] btrfs qgroup destroy 0/60350 '/run/timeshift/105175/backup' [21:12:38] E: Failed to destroy qgroup: '0/60350' [21:12:38] E: Failed to remove snapshot: 2026-04-23_21-58-52
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 3 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful. A full "btrfs check" is strongly recommended.
Quoted 2 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.
Quoted 6 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
Thanks,
Qu
Quoted 3 lines
Thank you.
I have the run the check command, which reported a variety of errors. The output is attached.
Are any recommendations available to attempt restoring the volume?
Thanks.
Quoted 24 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 3 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful. A full "btrfs check" is strongly recommended.
Quoted 2 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.Quoted 3 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Quoted 13 lines
Quoted 3 lines
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
Thanks,
QuQuoted 3 lines
Thank you.
[1/8] checking log skipped (none written)
[2/8] checking root items
[3/8] checking extents
super bytes used 743892983808 mismatches actual used 740558856192
ERROR: errors found in extent allocation tree or chunk allocation [4/8] checking free space tree
[5/8] checking fs roots
root 6855 inode 16523863 errors 400, nbytes wrong
root 58570 inode 16523863 errors 400, nbytes wrong
root 59486 inode 16523863 errors 400, nbytes wrong
root 60333 inode 16523863 errors 400, nbytes wrong
root 60367 inode 16523863 errors 400, nbytes wrong
root 60377 inode 16523863 errors 400, nbytes wrong
root 60383 inode 16523863 errors 400, nbytes wrong
root 60475 inode 16523863 errors 400, nbytes wrong
root 60713 inode 16523863 errors 400, nbytes wrong
root 61351 inode 16523863 errors 400, nbytes wrong
root 61421 inode 16523863 errors 400, nbytes wrong
root 61423 inode 16523863 errors 400, nbytes wrong
root 61425 inode 16523863 errors 400, nbytes wrong
root 61427 inode 16523863 errors 400, nbytes wrong
root 61429 inode 16523863 errors 400, nbytes wrong
root 61431 inode 16523863 errors 400, nbytes wrong
ERROR: errors found in fs roots
Opening filesystem to check...
Checking filesystem on /dev/nvme0n1p5
UUID: bbac86e5-eaba-45bf-bbaa-c2494e11831a
found 740558647296 bytes used, error(s) found
total csum bytes: 473596236
total tree bytes: 6834847744
total fs tree bytes: 5660327936
total extent tree bytes: 529825792
btree space waste bytes: 1514685920
file data blocks allocated: 11027378925568
referenced 886704037888
Quoted 3 lines · 在 2026/4/28 06:02, [email protected] 写道:
I have the run the check command, which reported a variety of errors. The output is attached.
Are any recommendations available to attempt restoring the volume?
The super block bytes mismatch is a minor one, which shouldn't affect normal operations.
But still you can use "btrfs rescue fix-device-size" to fix the problem.
The nbytes wrong can be minor too, but it's affecting all snapshots containing the inode 16523863.
You can either fix it by copying the inode to another location (can be inside the same btrfs), remove the original file, mv back the new copy. This will need to be done for every snapshot.
Or you can try "btrfs check --repair", which will do an in-place fix, but will still break the shared blocks of every snapshot.
Overall, I'd strongly recommend to remove all unused snapshots first before doing either fix.
Quoted 33 lines
Thanks.
Quoted 29 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 3 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful. A full "btrfs check" is strongly recommended.
Quoted 2 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.Quoted 6 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
So your fsck is mostly fine, the lag part is highly possible to be caused by qgroup.
If you do not need it, just disable it for good.
Thanks,
Qu
Quoted 9 lines
Quoted 8 lines
Thanks,
QuQuoted 3 lines
Thank you.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · 在 2026/4/28 08:17, [email protected] 写道:
Currently, scrub operations are not completing properly, so I certainly think it is important to try to repair the volume.
Which error do you expect is related to the particular problem, concerning scrub?
Can you provide the full "btrfs scrub start -BR" output?
Quoted 2 lines
Is there any evidence that data may have been lost,
Not yet.
Quoted 1 line
or concern that 'fix-device-size' could cause further loss?
That should be mostly safe, but if you're concerned about it, please update to a newer version of btrfs-progs.
Ubuntu is pretty bad at backporting fixes for btrfs-progs.
Quoted 1 line
Should the operation be done while the device is mounted, or only while not mounted?
Must be unmounted for btrfs check and btrfs rescue.
Thanks,
Qu
Quoted 78 lines
Thanks.
Quoted 72 lines · Qu Wenruo
Quoted 3 lines · 在 2026/4/28 06:02, [email protected] 写道:
I have the run the check command, which reported a variety of errors. The output is attached.
Are any recommendations available to attempt restoring the volume?
The super block bytes mismatch is a minor one, which shouldn't affect normal operations.
But still you can use "btrfs rescue fix-device-size" to fix the problem.
The nbytes wrong can be minor too, but it's affecting all snapshots containing the inode 16523863.
You can either fix it by copying the inode to another location (can be inside the same btrfs), remove the original file, mv back the new copy. This will need to be done for every snapshot.
Or you can try "btrfs check --repair", which will do an in-place fix, but will still break the shared blocks of every snapshot.
Overall, I'd strongly recommend to remove all unused snapshots first before doing either fix.
Quoted 33 lines
Thanks.
Quoted 29 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 3 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful. A full "btrfs check" is strongly recommended.
Quoted 2 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.Quoted 6 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
So your fsck is mostly fine, the lag part is highly possible to be caused by qgroup.
If you do not need it, just disable it for good.
Thanks,
QuQuoted 9 lines
Quoted 8 lines
Thanks,
QuQuoted 3 lines
Thank you.
Just as I was scanning for the name of the particular inode, the volume reverted to again insist on being without any free space.
I was able to extract the log for a new scrub, as requested (begun after the switch to being unable to create new files).
Thanks.
"Qu Wenruo" [email protected] – April 27, 2026 6:58 PM
Quoted 126 lines · 在 2026/4/28 08:17, [email protected] 写道:
Quoted 5 lines
Currently, scrub operations are not completing properly, so I certainly think it is important to try to repair the volume.
Which error do you expect is related to the particular problem, concerning scrub?
Can you provide the full "btrfs scrub start -BR" output?Quoted 2 lines
Is there any evidence that data may have been lost,
Not yet.Quoted 2 lines
or concern that
'fix-device-size' could cause further loss?
That should be mostly safe, but if you're concerned about it, please update to a newer version of btrfs-progs.
Ubuntu is pretty bad at backporting fixes for btrfs-progs.Quoted 2 lines
Should the operation be done
while the device is mounted, or only while not mounted?
Must be unmounted for btrfs check and btrfs rescue.
Thanks,
QuQuoted 96 lines
Thanks.Quoted 91 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/28 06:02, [email protected] 写道:
I have the run the check command, which reported a variety of errors. The output is attached.
Are any recommendations available to attempt restoring the volume?
The super block bytes mismatch is a minor one, which shouldn't affect normal operations.
But still you can use "btrfs rescue fix-device-size" to fix the problem.
The nbytes wrong can be minor too, but it's affecting all snapshots containing the inode 16523863.
You can either fix it by copying the inode to another location (can be inside the same btrfs), remove the original file, mv back the new copy. This will need to be done for every snapshot.
Or you can try "btrfs check --repair", which will do an in-place fix, but will still break the shared blocks of every snapshot.
Overall, I'd strongly recommend to remove all unused snapshots first before doing either fix.
Quoted 50 lines
Thanks.
Quoted 45 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 4 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful.
A full "btrfs check" is strongly recommended.Quoted 6 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.Quoted 11 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
So your fsck is mostly fine, the lag part is highly possible to be caused by qgroup.
If you do not need it, just disable it for good.
Thanks,
QuQuoted 5 lines
Quoted 5 lines
Thanks,
QuQuoted 1 line
Quoted 9 lines
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
Thank you.
WARNING: failed to open the progress status socket at /var/lib/btrfs/scrub.progress.bbac86e5-eaba-45bf-bbaa-c2494e11831a: No space left on device. Progress cannot be queried
WARNING: failed to write the progress status file: No space left on device. Status recording disabled
Starting scrub on devid 1
scrub done for bbac86e5-eaba-45bf-bbaa-c2494e11831a
Scrub started: Mon Apr 27 19:25:47 2026 Status: finished Duration: 0:03:34 data_extents_scrubbed: 174742 tree_extents_scrubbed: 1117169 data_bytes_scrubbed: 11444801536 tree_bytes_scrubbed: 18303696896 read_errors: 0 csum_errors: 0 verify_errors: 0 no_csum: 1621176 csum_discards: 0 super_errors: 0 malloc_errors: 0 uncorrectable_errors: 0 unverified_errors: 0 corrected_errors: 0 last_physical: 844727582720
Quoted 3 lines · brainchild
Just as I was scanning for the name of the particular inode, the volume reverted to again insist on being without any free space.
I was able to extract the log for a new scrub, as requested (begun after the switch to being unable to create new files).
Only 10GiB of data is scrubbed, meanwhile your data should be 600GiB+.
You mentioned that there is a swap file, how large is that file, and have you tried to disable the swap file before scrubbing?
Thanks,
Qu
Quoted 139 lines
Thanks.
"Qu Wenruo" [email protected] – April 27, 2026 6:58 PM
Quoted 135 lines · 在 2026/4/28 08:17, [email protected] 写道:
Quoted 5 lines
Currently, scrub operations are not completing properly, so I certainly think it is important to try to repair the volume.
Which error do you expect is related to the particular problem, concerning scrub?
Can you provide the full "btrfs scrub start -BR" output?
Quoted 2 lines
Is there any evidence that data may have been lost,
Not yet.
Quoted 2 lines
or concern that
'fix-device-size' could cause further loss?That should be mostly safe, but if you're concerned about it, please update to a newer version of btrfs-progs.
Ubuntu is pretty bad at backporting fixes for btrfs-progs.
Quoted 2 lines
Should the operation be done
while the device is mounted, or only while not mounted?Must be unmounted for btrfs check and btrfs rescue.
Thanks,
QuQuoted 103 lines
Thanks.
Quoted 96 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/28 06:02, [email protected] 写道:
I have the run the check command, which reported a variety of errors. The output is attached.
Are any recommendations available to attempt restoring the volume?
The super block bytes mismatch is a minor one, which shouldn't affect normal operations.
But still you can use "btrfs rescue fix-device-size" to fix the problem.
The nbytes wrong can be minor too, but it's affecting all snapshots containing the inode 16523863.
You can either fix it by copying the inode to another location (can be inside the same btrfs), remove the original file, mv back the new copy. This will need to be done for every snapshot.
Or you can try "btrfs check --repair", which will do an in-place fix, but will still break the shared blocks of every snapshot.
Overall, I'd strongly recommend to remove all unused snapshots first before doing either fix.
Quoted 50 lines
Thanks.
Quoted 45 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 4 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful.
A full "btrfs check" is strongly recommended.Quoted 6 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.Quoted 11 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
So your fsck is mostly fine, the lag part is highly possible to be caused by qgroup.
If you do not need it, just disable it for good.
Thanks,
QuQuoted 9 lines
Quoted 8 lines
Thanks,
QuQuoted 3 lines
Thank you.
I have not tried scrubbing without the swap file being in use.
I could try, but as the file is only 32Gb, I would be very surprised if it made any difference.
"Qu Wenruo" [email protected] – April 27, 2026 9:16 PM
Quoted 154 lines · brainchild
Quoted 3 lines
Just as I was scanning for the name of the particular inode, the volume reverted to again insist on being without any free space.
I was able to extract the log for a new scrub, as requested (begun after the switch to being unable to create new files).
Only 10GiB of data is scrubbed, meanwhile your data should be 600GiB+.
You mentioned that there is a swap file, how large is that file, and have you tried to disable the swap file before scrubbing?
Thanks,
QuQuoted 139 lines
Thanks.
"Qu Wenruo" [email protected] – April 27, 2026 6:58 PMQuoted 135 lines · 在 2026/4/28 08:17, [email protected] 写道:
Quoted 5 lines
Currently, scrub operations are not completing properly, so I certainly think it is important to try to repair the volume.
Which error do you expect is related to the particular problem, concerning scrub?
Can you provide the full "btrfs scrub start -BR" output?Quoted 2 lines
Is there any evidence that data may have been lost,
Not yet.Quoted 2 lines
or concern that
'fix-device-size' could cause further loss?
That should be mostly safe, but if you're concerned about it, please update to a newer version of btrfs-progs.
Ubuntu is pretty bad at backporting fixes for btrfs-progs.Quoted 2 lines
Should the operation be done
while the device is mounted, or only while not mounted?
Must be unmounted for btrfs check and btrfs rescue.
Thanks,
QuQuoted 96 lines
Thanks.Quoted 91 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/28 06:02, [email protected] 写道:
I have the run the check command, which reported a variety of errors. The output is attached.
Are any recommendations available to attempt restoring the volume?
The super block bytes mismatch is a minor one, which shouldn't affect normal operations.
But still you can use "btrfs rescue fix-device-size" to fix the problem.
The nbytes wrong can be minor too, but it's affecting all snapshots containing the inode 16523863.
You can either fix it by copying the inode to another location (can be inside the same btrfs), remove the original file, mv back the new copy. This will need to be done for every snapshot.
Or you can try "btrfs check --repair", which will do an in-place fix, but will still break the shared blocks of every snapshot.
Overall, I'd strongly recommend to remove all unused snapshots first before doing either fix.
Quoted 50 lines
Thanks.
Quoted 45 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 4 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful.
A full "btrfs check" is strongly recommended.Quoted 6 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.Quoted 11 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
So your fsck is mostly fine, the lag part is highly possible to be caused by qgroup.
If you do not need it, just disable it for good.
Thanks,
QuQuoted 5 lines
Quoted 5 lines
Thanks,
QuQuoted 1 line
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
Thank you.
With swap disabled, the scrub operation seems to reach completion, with no errors found.
However, the check operation still discovers the same errors.
The output of `rescue fix-device-size' is "No device size related problem found".
After, check still reports the same errors, including the super mismatch.
"brainchild" [email protected] – April 28, 2026 1:21 AM
Quoted 161 lines
I have not tried scrubbing without the swap file being in use.
I could try, but as the file is only 32Gb, I would be very surprised if it made any difference.
"Qu Wenruo" [email protected] – April 27, 2026 9:16 PMQuoted 148 lines · brainchild
Quoted 3 lines
Just as I was scanning for the name of the particular inode, the volume reverted to again insist on being without any free space.
I was able to extract the log for a new scrub, as requested (begun after the switch to being unable to create new files).
Only 10GiB of data is scrubbed, meanwhile your data should be 600GiB+.
You mentioned that there is a swap file, how large is that file, and have you tried to disable the swap file before scrubbing?
Thanks,
QuQuoted 135 lines
Thanks.
"Qu Wenruo" [email protected] – April 27, 2026 6:58 PMQuoted 131 lines · 在 2026/4/28 08:17, [email protected] 写道:
Quoted 5 lines
Currently, scrub operations are not completing properly, so I certainly think it is important to try to repair the volume.
Which error do you expect is related to the particular problem, concerning scrub?
Can you provide the full "btrfs scrub start -BR" output?Quoted 2 lines
Is there any evidence that data may have been lost,
Not yet.Quoted 2 lines
or concern that
'fix-device-size' could cause further loss?
That should be mostly safe, but if you're concerned about it, please update to a newer version of btrfs-progs.
Ubuntu is pretty bad at backporting fixes for btrfs-progs.Quoted 2 lines
Should the operation be done
while the device is mounted, or only while not mounted?
Must be unmounted for btrfs check and btrfs rescue.
Thanks,
QuQuoted 101 lines
Thanks.Quoted 96 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/28 06:02, [email protected] 写道:
I have the run the check command, which reported a variety of errors. The output is attached.
Are any recommendations available to attempt restoring the volume?
The super block bytes mismatch is a minor one, which shouldn't affect normal operations.
But still you can use "btrfs rescue fix-device-size" to fix the problem.
The nbytes wrong can be minor too, but it's affecting all snapshots containing the inode 16523863.
You can either fix it by copying the inode to another location (can be inside the same btrfs), remove the original file, mv back the new copy. This will need to be done for every snapshot.
Or you can try "btrfs check --repair", which will do an in-place fix, but will still break the shared blocks of every snapshot.
Overall, I'd strongly recommend to remove all unused snapshots first before doing either fix.
Quoted 50 lines
Thanks.
Quoted 45 lines · Qu Wenruo
Quoted 4 lines · 在 2026/4/27 09:22, [email protected] 写道:
Hello.
I am struggling with a poorly behaved BTRFS volume.
[...]
Quoted 4 lines
---
No errors are reported in the kernel log, only warnings about skipping the swap file during scrub.
If you assume the fs has some corruption, none of the above is really useful.
A full "btrfs check" is strongly recommended.Quoted 6 lines
Second, within the logs generated for Timeshift, a concerning pattern recurs, as in the attached example. Further, during the periods in which are generated logs such as the one attached, the entire system lags considerably. It is clear that the volume is not healthy.
The lag is mostly caused by qgroup.
You have a lot of snapshots (shown by the super large snapshot id), every time a large snapshot/subvolume is deleted, btrfs will try to disable qgroup to avoid such lag, but if whatever script/tool decides to rescan qgroup when the snapshot/subvolume deleting is still under going, the lag will be re-introduced.Quoted 11 lines
I was using a recent 6.x kernel, I believe one of 6.18.x, when the problem emerged. I upgraded by to 7.0, finding no improvement in the operation of the volume.
Also, I tried initiating the scrub through the most recent static build of the user-space utility (i.e. btrfs-progs), with no improvement.
I would like some suggestions for restoring the volume to health, to avoid the need to provision a new volume from scratch.
"btrfs check" first, if no error, disable qgroup if you have frequent snapshot creation/deletion.
So your fsck is mostly fine, the lag part is highly possible to be caused by qgroup.
If you do not need it, just disable it for good.
Thanks,
QuQuoted 9 lines
Quoted 8 lines
Thanks,
QuQuoted 3 lines
Thank you.
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 3 lines · brainchild
With swap disabled, the scrub operation seems to reach completion, with no errors found.
However, the check operation still discovers the same errors.
Scrub is not a fsck, from man page of btrfs-scrub:
Note:
Scrub is not a filesystem checker (fsck, btrfs-check(8)). It
can only detect filesystem damage using the checksum validation, and it can only repair filesystem damage by copying from other known good replicas.
btrfs-check(8) performs more exhaustive checking and can
sometimes be used, with expert guidance, to rebuild certain corrupted filesystem structures in the absence of any good replica.
Quoted 4 lines
The output of `rescue fix-device-size' is "No device size related problem found".
After, check still reports the same errors, including the super mismatch.
If you do not want to manually fix the nbytes mismatch, go "btrfs check --repair", which may also help to fix the super block size mismatch.
I am simply reporting on my observations.
Deleting all instances of the file corresponding to the identified inode, across all subvolumes, has resolved the problem with nbytes, but fix-device-size reported no action, and the super mismatch remains.
In case you have any further thoughts on finding a resolution, I am eager for any suggestions. I would like the volume to be fully healthy.
Quoted 22 lines · Qu Wenruo
Quoted 2 lines · brainchild
With swap disabled, the scrub operation seems to reach completion, with no errors found.
However, the check operation still discovers the same errors.Scrub is not a fsck, from man page of btrfs-scrub:
Note:
Scrub is not a filesystem checker (fsck, btrfs-check(8)). It
can only detect filesystem damage using the checksum validation, and it can only repair filesystem damage by copying from other known good replicas.btrfs-check(8) performs more exhaustive checking and can
sometimes be used, with expert guidance, to rebuild certain corrupted filesystem structures in the absence of any good replica.Quoted 4 lines
The output of `rescue fix-device-size' is "No device size related
problem found".
After, check still reports the same errors, including the super
mismatch.If you do not want to manually fix the nbytes mismatch, go "btrfs check --repair", which may also help to fix the super block size mismatch.
Quoted 5 lines · 在 2026/4/28 13:33, [email protected] 写道:
I am simply reporting on my observations.
Deleting all instances of the file corresponding to the identified inode, across all subvolumes, has resolved the problem with nbytes, but fix-device-size reported no action, and the super mismatch remains.
In case you have any further thoughts on finding a resolution, I am eager for any suggestions. I would like the volume to be fully healthy.
In this case, your fs has no problem and can be used as usual. That super block bytes mismatch can be ignored.
I'll enhance the rescue command to handle the case you reported.
Thanks,
Qu
Quoted 23 lines · Qu Wenruo
Quoted 18 lines · brainchild
Quoted 2 lines
With swap disabled, the scrub operation seems to reach completion, with no errors found. However, the check operation still discovers the same errors.
Scrub is not a fsck, from man page of btrfs-scrub:
Note:
Scrub is not a filesystem checker (fsck, btrfs-check(8)). It can only detect filesystem damage using the checksum validation, and it can only repair filesystem damage by copying from other known good replicas.btrfs-check(8) performs more exhaustive checking and can sometimes be used, with expert guidance, to rebuild certain corrupted filesystem structures in the absence of any good replica.
Quoted 2 lines
The output of `rescue fix-device-size' is "No device size related problem found". After, check still reports the same errors, including the super mismatch.
If you do not want to manually fix the nbytes mismatch, go "btrfs check --repair", which may also help to fix the super block size mismatch.
The volume seems to suffer still from at least two problems:
- If scrub actually completes under any circumstances, it is required that the swap file is unused. Disabling swap is not always feasible during normal operation of the system.
- Periodically, the filesystem reverts to a state in which creation of new files is prevented. Naturally, it is impossible to use a system normally if the root filesystem is effectively read only.
As such, I feel less optimistic than you about the usability of the volume.
Apr 28, 2026 01:13:47 Qu Wenruo <[email protected]>:
Quoted 36 lines · 在 2026/4/28 13:33, [email protected] 写道:
Quoted 3 lines
I am simply reporting on my observations.
Deleting all instances of the file corresponding to the identified inode, across all subvolumes, has resolved the problem with nbytes, but fix-device-size reported no action, and the super mismatch remains. In case you have any further thoughts on finding a resolution, I am eager for any suggestions. I would like the volume to be fully healthy.In this case, your fs has no problem and can be used as usual. That super block bytes mismatch can be ignored.
I'll enhance the rescue command to handle the case you reported.
Thanks,
QuQuoted 21 lines · Qu Wenruo
Quoted 18 lines · brainchild
Quoted 2 lines
With swap disabled, the scrub operation seems to reach completion, with no errors found. However, the check operation still discovers the same errors.
Scrub is not a fsck, from man page of btrfs-scrub:
Note:
Scrub is not a filesystem checker (fsck, btrfs-check(8)). It can only detect filesystem damage using the checksum validation, and it can only repair filesystem damage by copying from other known good replicas.btrfs-check(8) performs more exhaustive checking and can sometimes be used, with expert guidance, to rebuild certain corrupted filesystem structures in the absence of any good replica.
Quoted 2 lines
The output of `rescue fix-device-size' is "No device size related problem found". After, check still reports the same errors, including the super mismatch.
If you do not want to manually fix the nbytes mismatch, go "btrfs check --repair", which may also help to fix the super block size mismatch.
Quoted 3 lines · 在 2026/4/28 14:59, [email protected] 写道:
The volume seems to suffer still from at least two problems:
- If scrub actually completes under any circumstances, it is required that the swap file is unused. Disabling swap is not always feasible during normal operation of the system.
By design it's not easy to support swap file on a COW filesystem, that's why btrfs has so many limits when using swap files.
I strongly don't recommend to use swap files on btrfs, as you have already experienced the limit on scrub, and I believe a lot of end users are not aware of all the limits when using swap file on btrfs, please check the long long list of limitations in "SWAPFILE SUPPORT" of btrfs(5).
Quoted 2 lines
- Periodically, the filesystem reverts to a state in which creation of new files is prevented. Naturally, it is impossible to use a system normally if the root filesystem is effectively read only.
Any dmesg of that RO flips? That indicates the fs flipped read-only, which is a huge problem by itself.
Especially with your initial info, there should be enough data space, metadata space is less ideal but should be enough.
$ sudo btrfs filesystem df / Data, single: total=813.13GiB, used=662.49GiB System, DUP: total=32.00MiB, used=144.00KiB Metadata, DUP: total=9.03GiB, used=6.26GiB GlobalReserve, single: total=512.00MiB, used=0.00B
Considering how many snapshots you have (triggering qgroup lag), I strongly recommended to remove unused snapshots to free up space.
After freeing up enough space, then try to balance data block groups to make space for future metadata usages.
Quoted 2 lines · Qu Wenruo
I strongly don't recommend to use swap files on btrfs, as you have already experienced the limit on scrub, and I believe a lot of end users are not aware of all the limits when using swap file on btrfs, please check the long long list of limitations in "SWAPFILE SUPPORT" of btrfs(5).
Is it expected that the scrub operation cannot function properly if the volume has a swap file? I never before observed such a problem, nor find any mention in the documentation.
The specific restrictions, as documented, for the swap file, seem completely compatible with my use, a single partition with no data duplication. I have no need for spanning devices or duplicating data, on the particular system.
Quoted 1 line
Any dmesg of that RO flips? That indicates the fs flipped read-only, which is a huge problem by itself.
No. There are no kernel messages that are errors for the file system, or switches to read-only.
Quoted 1 line
Especially with your initial info, there should be enough data space, metadata space is less ideal but should be enough.
I have read that the space allocated for metadata is expanded as needed. Why would problems follow from too little space being allocated?
Quoted 3 lines
Considering how many snapshots you have (triggering qgroup lag), I strongly recommended to remove unused snapshots to free up space.
After freeing up enough space, then try to balance data block groups to make space for future metadata usages.
The situation with balance is quite confused.
The problem with the reported lack of free space first occurred several weeks ago. At that time, I deleted snapshots, and ran balance operations with incrementally higher usage values for data and metadata. By the end, I had run the operation, without reported failure, with values as high as 95%. Normally, such an operation would be very long, but in my case it finished in less than a minute. Also by the end, only about ten blocks in total had actually been reported as moved.
Perhaps my installation of btrfsd has been successfully maintaining the balance for the volume. The system logs are not extensive enough for me to know when it last performed any operations.
Regardless, it seems that the general problems are not becoming resolved by invocations of balance.
Further to the comments in the previous message, I have also found some messages in the kernel log, from the balance operations, which may be relevant.
As the console output of the command is "ERROR: error during balancing '/': No space left on device", the kernel messages are as shown below.
---
balance: start -musage=50 -susage=50
BTRFS info (device nvme0n1p5): relocating block group 4126692868096 flags metadata|dup BTRFS info (device nvme0n1p5): found 9279 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4126155997184 flags metadata|dup BTRFS info (device nvme0n1p5): found 6365 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4125149364224 flags metadata|dup BTRFS info (device nvme0n1p5): found 10145 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4124612493312 flags metadata|dup BTRFS info (device nvme0n1p5): found 11487 extents, stage: move data extents BTRFS info (device nvme0n1p5): 1 enospc errors during balance
BTRFS info (device nvme0n1p5): balance: ended with status: -28
Quoted 29 lines · On Tue, Apr 28 2026 at 03:30:00 PM -04:00:00, [email protected] wrote:
Quoted 2 lines · Qu Wenruo
I strongly don't recommend to use swap files on btrfs, as you have already experienced the limit on scrub, and I believe a lot of end users are not aware of all the limits when using swap file on btrfs, please check the long long list of limitations in "SWAPFILE SUPPORT" of btrfs(5).
Is it expected that the scrub operation cannot function properly if the volume has a swap file? I never before observed such a problem, nor find any mention in the documentation.
The specific restrictions, as documented, for the swap file, seem completely compatible with my use, a single partition with no data duplication. I have no need for spanning devices or duplicating data, on the particular system.
Quoted 1 line
Any dmesg of that RO flips? That indicates the fs flipped read-only, which is a huge problem by itself.
No. There are no kernel messages that are errors for the file system, or switches to read-only.
Quoted 1 line
Especially with your initial info, there should be enough data space, metadata space is less ideal but should be enough.
I have read that the space allocated for metadata is expanded as needed. Why would problems follow from too little space being allocated?
Quoted 3 lines
Considering how many snapshots you have (triggering qgroup lag), I strongly recommended to remove unused snapshots to free up space.
After freeing up enough space, then try to balance data block groups to make space for future metadata usages.
The situation with balance is quite confused.
The problem with the reported lack of free space first occurred several weeks ago. At that time, I deleted snapshots, and ran balance operations with incrementally higher usage values for data and metadata. By the end, I had run the operation, without reported failure, with values as high as 95%. Normally, such an operation would be very long, but in my case it finished in less than a minute. Also by the end, only about ten blocks in total had actually been reported as moved.
Perhaps my installation of btrfsd has been successfully maintaining the balance for the volume. The system logs are not extensive enough for me to know when it last performed any operations.
Regardless, it seems that the general problems are not becoming resolved by invocations of balance.
Quoted 6 lines · 在 2026/4/29 07:49, [email protected] 写道:
Further to the comments in the previous message, I have also found some messages in the kernel log, from the balance operations, which may be relevant.
As the console output of the command is "ERROR: error during balancing '/': No space left on device", the kernel messages are as shown below.
---
Quoted 11 lines
balance: start -musage=50 -susage=50
BTRFS info (device nvme0n1p5): relocating block group 4126692868096 flags metadata|dup BTRFS info (device nvme0n1p5): found 9279 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4126155997184 flags metadata|dup BTRFS info (device nvme0n1p5): found 6365 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4125149364224 flags metadata|dup BTRFS info (device nvme0n1p5): found 10145 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4124612493312 flags metadata|dup BTRFS info (device nvme0n1p5): found 11487 extents, stage: move data extents BTRFS info (device nvme0n1p5): 1 enospc errors during balance
BTRFS info (device nvme0n1p5): balance: ended with status: -28
From the dmesg, you're relocating only metadata block groups.
Meanwhile all the free space is inside data block groups, you need to balance *only* data block groups to free up space for metadata.
Not the opposite.
Quoted 35 lines · On Tue, Apr 28 2026 at 03:30:00 PM -04:00:00, [email protected] wrote:
Quoted 29 lines · Qu Wenruo
Quoted 2 lines
I strongly don't recommend to use swap files on btrfs, as you have already experienced the limit on scrub, and I believe a lot of end users are not aware of all the limits when using swap file on btrfs, please check the long long list of limitations in "SWAPFILE SUPPORT" of btrfs(5).
Is it expected that the scrub operation cannot function properly if the volume has a swap file? I never before observed such a problem, nor find any mention in the documentation.
The specific restrictions, as documented, for the swap file, seem completely compatible with my use, a single partition with no data duplication. I have no need for spanning devices or duplicating data, on the particular system.
Quoted 1 line
Any dmesg of that RO flips? That indicates the fs flipped read-only, which is a huge problem by itself.
No. There are no kernel messages that are errors for the file system, or switches to read-only.
Quoted 1 line
Especially with your initial info, there should be enough data space, metadata space is less ideal but should be enough.
I have read that the space allocated for metadata is expanded as needed. Why would problems follow from too little space being allocated?
Quoted 3 lines
Considering how many snapshots you have (triggering qgroup lag), I strongly recommended to remove unused snapshots to free up space.
After freeing up enough space, then try to balance data block groups to make space for future metadata usages.
The situation with balance is quite confused.
The problem with the reported lack of free space first occurred several weeks ago. At that time, I deleted snapshots, and ran balance operations with incrementally higher usage values for data and metadata. By the end, I had run the operation, without reported failure, with values as high as 95%. Normally, such an operation would be very long, but in my case it finished in less than a minute. Also by the end, only about ten blocks in total had actually been reported as moved.
Perhaps my installation of btrfsd has been successfully maintaining the balance for the volume. The system logs are not extensive enough for me to know when it last performed any operations.
Regardless, it seems that the general problems are not becoming resolved by invocations of balance.
Quoted 21 lines · Qu Wenruo
Quoted 6 lines · 在 2026/4/29 07:49, [email protected] 写道:
Further to the comments in the previous message, I have also found some messages in the kernel log, from the balance operations, which may be relevant.
As the console output of the command is "ERROR: error during balancing '/': No space left on device", the kernel messages are as shown below.
---
Quoted 11 lines
balance: start -musage=50 -susage=50
BTRFS info (device nvme0n1p5): relocating block group 4126692868096 flags metadata|dup BTRFS info (device nvme0n1p5): found 9279 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4126155997184 flags metadata|dup BTRFS info (device nvme0n1p5): found 6365 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4125149364224 flags metadata|dup BTRFS info (device nvme0n1p5): found 10145 extents, stage: move data extents BTRFS info (device nvme0n1p5): relocating block group 4124612493312 flags metadata|dup BTRFS info (device nvme0n1p5): found 11487 extents, stage: move data extents BTRFS info (device nvme0n1p5): 1 enospc errors during balance
BTRFS info (device nvme0n1p5): balance: ended with status: -28
Quoted 5 lines
From the dmesg, you're relocating only metadata block groups.
Meanwhile all the free space is inside data block groups, you need to balance *only* data block groups to free up space for metadata.
Not the opposite.
And forgot to mention, balance is also affected by swapfiles.
The same as scrub, a block group (1GiB) will be completely skipped if there is any extent of an active swapfile.
So your previous balance runs may also be screwed up by your swapfile.
Quoted 38 lines
Quoted 36 lines · On Tue, Apr 28 2026 at 03:30:00 PM -04:00:00, [email protected] wrote:
Quoted 29 lines · Qu Wenruo
Quoted 2 lines
I strongly don't recommend to use swap files on btrfs, as you have already experienced the limit on scrub, and I believe a lot of end users are not aware of all the limits when using swap file on btrfs, please check the long long list of limitations in "SWAPFILE SUPPORT" of btrfs(5).
Is it expected that the scrub operation cannot function properly if the volume has a swap file? I never before observed such a problem, nor find any mention in the documentation.
The specific restrictions, as documented, for the swap file, seem completely compatible with my use, a single partition with no data duplication. I have no need for spanning devices or duplicating data, on the particular system.
Quoted 1 line
Any dmesg of that RO flips? That indicates the fs flipped read-only, which is a huge problem by itself.
No. There are no kernel messages that are errors for the file system, or switches to read-only.
Quoted 1 line
Especially with your initial info, there should be enough data space, metadata space is less ideal but should be enough.
I have read that the space allocated for metadata is expanded as needed. Why would problems follow from too little space being allocated?
Quoted 3 lines
Considering how many snapshots you have (triggering qgroup lag), I strongly recommended to remove unused snapshots to free up space.
After freeing up enough space, then try to balance data block groups to make space for future metadata usages.
The situation with balance is quite confused.
The problem with the reported lack of free space first occurred several weeks ago. At that time, I deleted snapshots, and ran balance operations with incrementally higher usage values for data and metadata. By the end, I had run the operation, without reported failure, with values as high as 95%. Normally, such an operation would be very long, but in my case it finished in less than a minute. Also by the end, only about ten blocks in total had actually been reported as moved.
Perhaps my installation of btrfsd has been successfully maintaining the balance for the volume. The system logs are not extensive enough for me to know when it last performed any operations.
Regardless, it seems that the general problems are not becoming resolved by invocations of balance.
Quoted 6 lines · 在 2026/4/29 05:00, [email protected] 写道:
Quoted 2 lines · Qu Wenruo
I strongly don't recommend to use swap files on btrfs, as you have already experienced the limit on scrub, and I believe a lot of end users are not aware of all the limits when using swap file on btrfs, please check the long long list of limitations in "SWAPFILE SUPPORT" of btrfs(5).
Is it expected that the scrub operation cannot function properly if the volume has a swap file?
Yes, scrub works by iterating each block group, and to avoid concurrency modification, scrub will mark the block group RO.
But if there is a scrub covering even one block of the block group, the block group cannot be marked read-only, thus scrub will skip the whole block group.
Quoted 1 line
I never before observed such a problem, nor find any mention in the documentation.
OK, btrfs(5) only mentions dev-replace, not scrub itself.
Something we need to update it.
Quoted 6 lines
The specific restrictions, as documented, for the swap file, seem completely compatible with my use, a single partition with no data duplication. I have no need for spanning devices or duplicating data, on the particular system.
Quoted 1 line
Any dmesg of that RO flips? That indicates the fs flipped read-only, which is a huge problem by itself.
No. There are no kernel messages that are errors for the file system, or switches to read-only.
Then how did the problem of failing to create new files happen?
Any extra output when that happened?
Just returning -ENOSPC error messages but the fs is still read-write?
Then it may be not enough meta/data space left.
Quoted 4 lines
Quoted 2 lines
Especially with your initial info, there should be enough data space, metadata space is less ideal but should be enough.
I have read that the space allocated for metadata is expanded as needed. Why would problems follow from too little space being allocated?
Because you have no unallocated space so metadata can not be expanded anymore.
Quoted 8 lines
Quoted 4 lines
Considering how many snapshots you have (triggering qgroup lag), I strongly recommended to remove unused snapshots to free up space.
After freeing up enough space, then try to balance data block groups to make space for future metadata usages.
The situation with balance is quite confused.
The problem with the reported lack of free space first occurred several weeks ago. At that time, I deleted snapshots, and ran balance operations with incrementally higher usage values for data and metadata. By the end, I had run the operation, without reported failure, with values as high as 95%. Normally, such an operation would be very long, but in my case it finished in less than a minute. Also by the end, only about ten blocks in total had actually been reported as moved.
But your 'btrfs fi df' shows the other way, a lot of free data space, but not much for metadata, and still no unallocated space.
Quoted 7 lines
Perhaps my installation of btrfsd has been successfully maintaining the balance for the volume. The system logs are not extensive enough for me to know when it last performed any operations.
Regardless, it seems that the general problems are not becoming resolved by invocations of balance.
Quoted 6 lines · Qu Wenruo
Quoted 3 lines · 在 2026/4/29 05:00, [email protected] 写道:
Quoted 1 line · Qu Wenruo
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
I strongly don't recommend to use swap files on btrfs, as you have already experienced the limit on scrub, and I believe a lot of end users are not aware of all the limits when using swap file on btrfs, please check the long long list of limitations in "SWAPFILE SUPPORT" of btrfs(5).
Is it expected that the scrub operation cannot function properly if the volume has a swap file?
Yes, scrub works by iterating each block group, and to avoid concurrency modification, scrub will mark the block group RO.
But if there is a scrub covering even one block of the block group,
Typo, "scrub" -> "swap file".
Quoted 49 lines
the block group cannot be marked read-only, thus scrub will skip the whole block group.
Quoted 1 line
I never before observed such a problem, nor find any mention in the documentation.
OK, btrfs(5) only mentions dev-replace, not scrub itself.
Something we need to update it.
Quoted 6 lines
The specific restrictions, as documented, for the swap file, seem completely compatible with my use, a single partition with no data duplication. I have no need for spanning devices or duplicating data, on the particular system.
Quoted 1 line
Any dmesg of that RO flips? That indicates the fs flipped read-only, which is a huge problem by itself.
No. There are no kernel messages that are errors for the file system, or switches to read-only.
Then how did the problem of failing to create new files happen?
Any extra output when that happened?
Just returning-ENOSPCerror messages but the fs is still read-write?Then it may be not enough meta/data space left.
Quoted 4 lines
Quoted 2 lines
Especially with your initial info, there should be enough data space, metadata space is less ideal but should be enough.
I have read that the space allocated for metadata is expanded as needed. Why would problems follow from too little space being allocated?
Because you have no unallocated space so metadata can not be expanded anymore.
Quoted 8 lines
Quoted 4 lines
Considering how many snapshots you have (triggering qgroup lag), I strongly recommended to remove unused snapshots to free up space.
After freeing up enough space, then try to balance data block groups to make space for future metadata usages.
The situation with balance is quite confused.
The problem with the reported lack of free space first occurred several weeks ago. At that time, I deleted snapshots, and ran balance operations with incrementally higher usage values for data and metadata. By the end, I had run the operation, without reported failure, with values as high as 95%. Normally, such an operation would be very long, but in my case it finished in less than a minute. Also by the end, only about ten blocks in total had actually been reported as moved.
But your 'btrfs fi df' shows the other way, a lot of free data space, but not much for metadata, and still no unallocated space.
Quoted 7 lines
Perhaps my installation of btrfsd has been successfully maintaining the balance for the volume. The system logs are not extensive enough for me to know when it last performed any operations.
Regardless, it seems that the general problems are not becoming resolved by invocations of balance.
Quoted 3 lines · Qu Wenruo
Quoted 3 lines
Then how did the problem of failing to create new files happen?
It began suddenly, with no clear cause.
The Timeshift snapshots had been causing serious lag since a few weeks earlier, but investigating had not yet become a top priority.
One similarity for the two times the problem has occurred is that I was running file searches across the whole system using 'find'. Perhaps something about these searches triggered the more immediate problem.
Quoted 2 lines
Quoted 2 lines
Any extra output when that happened?
Just returning-ENOSPCerror messages but the fs is still read-write?
Yes, the FS is still RW.
Quoted 8 lines
Quoted 8 lines
Because you have no unallocated space so metadata can not be expanded anymore.
Meanwhile all the free space is inside data block groups, you need to balance *only* data block groups to free up space for metadata.
Not the opposite.
I already tried balancing just data, but there is no longer any benefit. It seems all of the data is already balanced. Otherwise, balance is failing to do its job.
Attempts to balance data now instantly complete, reporting nothing to do:
---
$ time sudo btrfs balance start -dusage=95 / Done, had to relocate 0 out of 833 chunks
real 0m0.059s
user 0m0.000s
sys 0m0.013s
---
The only hint of any problem is from the kernel log:
---_btrfs_printk: 788 callbacks suppressed
---
The other messages relate only to reporting blocks as skipped due to the swap file.
Earlier the more aggressive balance operations were failing, reporting no available space. From this situation, I started with -dusage=5, and then reached 95, in increments of 5. Each of the calls lasted no more than a few minutes, and in total only about ten blocks were reported as moved.
I would consider removing the swap file, but it would require shrinking the Btrfs partition, to make room for another partition dedicated to swap. I wonder whether it is safe to resize the volume, considering its unstable condition.
Quoted 1 line
Also, since the swap file is so small compared to the size of the volume, I doubt that it is causing the serious problems.
Quoted 44 lines · 在 2026/4/29 10:27, [email protected] 写道:
Quoted 3 lines · Qu Wenruo
Quoted 3 lines
Then how did the problem of failing to create new files happen?
It began suddenly, with no clear cause.
The Timeshift snapshots had been causing serious lag since a few weeks earlier, but investigating had not yet become a top priority.
One similarity for the two times the problem has occurred is that I was running file searches across the whole system using 'find'. Perhaps something about these searches triggered the more immediate problem.
Quoted 2 lines
Quoted 2 lines
Any extra output when that happened?
Just returning-ENOSPCerror messages but the fs is still read-write?Yes, the FS is still RW.
Quoted 8 lines
Quoted 8 lines
Because you have no unallocated space so metadata can not be expanded anymore.
Meanwhile all the free space is inside data block groups, you need to balance *only* data block groups to free up space for metadata.
Not the opposite.
I already tried balancing just data, but there is no longer any benefit. It seems all of the data is already balanced. Otherwise, balance is failing to do its job.
Attempts to balance data now instantly complete, reporting nothing to do:
---
$ time sudo btrfs balance start -dusage=95 / Done, had to relocate 0 out of 833 chunksreal 0m0.059s
user 0m0.000s
sys 0m0.013s
---The only hint of any problem is from the kernel log:
---
_btrfs_printk: 788 callbacks suppressed
---
Quoted 9 lines
The other messages relate only to reporting blocks as skipped due to the swap file.
Earlier the more aggressive balance operations were failing, reporting no available space. From this situation, I started with -dusage=5, and then reached 95, in increments of 5. Each of the calls lasted no more than a few minutes, and in total only about ten blocks were reported as moved.
I would consider removing the swap file, but it would require shrinking the Btrfs partition, to make room for another partition dedicated to swap. I wonder whether it is safe to resize the volume, considering its unstable condition.
Quoted 1 line
Also, since the swap file is so small compared to the size of the volume, I doubt that it is causing the serious problems.
Nope, just disable the swap file.
32GiB seems small, but you have no control on how large the real file extents are.
It can be 128MiB (the normal one), and you still have 256 extents.
If each extent is one a different block group, you can have 256GiB data blocks unable to be balanced.
Furthermore, your scrub is already showing that only 11GiB data can be properly scrubbed, the remaining hundreds of GiB are all skipped.
Now you tell me if this is causing series problems.
And this is the new docs update to explicitly warn end users about the problems with swap files on btrfs:
https://lore.kernel.org/linux-btrfs/357262c371343c6d6919b7827803194cb46a5e40.1777420050.git.wqu@suse.com/T/#u
Quoted 1 line · Qu Wenruo
Nope, just disable the swap file.
Is shrinking the volume safe, or even possible, in the current condition?
Quoted 6 lines · 在 2026/4/29 10:46, [email protected] 写道:
Quoted 1 line · Qu Wenruo
Nope, just disable the swap file.
Is shrinking the volume safe, or even possible, in the current condition?
Shrink is going to relocate some block groups, and if any block group is covered by a swap extent, the shrink will fail.
So I'm just asking to disable swap file, with all the explanation provided, then balance data.
If you do not want to follow, do whatever you want, you are on your own.
All I can do is to enhance the docs and hope there will be no more end users sticking with stupid swapfile idea without really understanding all the limitations.
Quoted 2 lines · Qu Wenruo
All I can do is to enhance the docs and hope there will be no more end users sticking with stupid swapfile idea without really understanding all the limitations.
I think we misunderstood each other. Running balance with swap disabled is not a problem. I have just done it, and as you predicted, many more blocks were identified for balancing.
However, I cannot disable swap permanently. If I am planning to migrate away from a swap file on the Btrfs volume, then I need to resize it to make space for a separate partition for swap. I was concerned that the super mismatch earlier discovered might make it unsafe to resize the partition.