LTP fs_fill test results in BTRFS warning (device loop0): could not allocate space for a delete; will truncate on mount warnings
Hello,
While running LTP [1] as part of CKI [2] testing, we noticed the fs_fill test fails pretty consistently with BTRFS warnings seen below, this is seen with recent kernel (5.2.12). I have included the logs below for reference.
Thank you,
Rachel
* https://artifacts.cki-project.org/pipelines/147783/logs/aarch64_host_1_console.log
8747.237733] LTP: starting fs_fill
[ 8748.085884] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem [ 8748.103546] EXT4-fs (loop0): mounted filesystem without journal. Opts: (null) [ 8751.856179] EXT4-fs (loop0): mounting ext3 file system using the ext4 subsystem [ 8751.882784] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8759.496036] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8768.360525] XFS (loop0): Mounting V5 Filesystem [ 8768.368464] XFS (loop0): Ending clean mount [ 8775.694012] XFS (loop0): Unmounting Filesystem [ 8776.179816] BTRFS: device fsid 3f8f3f1f-6293-40c4-91f4-bfe1d5c4451d devid 1 transid 5 /dev/loop0 [ 8776.190118] BTRFS info (device loop0): disk space caching is enabled [ 8776.195513] BTRFS info (device loop0): has skinny extents [ 8776.200927] BTRFS info (device loop0): flagging fs with big metadata feature [ 8776.212841] BTRFS info (device loop0): checking UUID tree [ 9062.793701] BTRFS warning (device loop0): could not allocate space for a delete; will truncate on mount [ 9062.794100] BTRFS warning (device loop0): could not allocate space for a delete; will truncate on mount <snip> [ 9062.794172] BTRFS warning (device loop0): could not allocate space for a delete; will truncate on mount
fs_fill log: https://paste.fedoraproject.org/paste/368EirX-AHSsZhG4Zutm2w
[1] https://github.com/linux-test-project/ltp
[2] https://cki-project.org/
<https://projects.engineering.redhat.com/secure/ViewProfile.jspa?name=jstancek>
Quoted 8 lines · Rachel Sibley
Hello,
While running LTP [1] as part of CKI [2] testing, we noticed the
fs_filltest fails pretty
consistently with BTRFS warnings seen below, this is seen with recent kernel (5.2.12).
I have included the logs below for reference.
I'm only 6 for 6, but so far can't get it to trigger with 5.3.0-rc7
Signature · 1 line
Chris Murphy
Quoted 5 lines · Chris Murphy
Quoted 3 lines · Rachel Sibley
Hello,
Quoted 7 lines
Quoted 5 lines
While running LTP [1] as part of CKI [2] testing, we noticed the
fs_filltest fails pretty
consistently with BTRFS warnings seen below, this is seen with recent kernel (5.2.12).
I have included the logs below for reference.I'm only 6 for 6, but so far can't get it to trigger with 5.3.0-rc7
Also cannot reproduce, 10 for 10 with 5.2.13-200.fc30.x86_64 which was just built in koji, same as 5.2.12 which was quickly replaced with a single unrelated revert for touchpads.
But comparing fs_fill logs, yours has a ton more writer threads (~6x) and ENOSPC counts. Possibly related?
Signature · 1 line
Chris Murphy
Quoted 9 lines · Chris Murphy
Quoted 8 lines · Chris Murphy
Quoted 7 lines · Rachel Sibley
Hello,
While running LTP [1] as part of CKI [2] testing, we noticed the
fs_filltest fails pretty
consistently with BTRFS warnings seen below, this is seen with recent kernel (5.2.12).
I have included the logs below for reference.
Quoted 7 lines
Quoted 1 line
I'm only 6 for 6, but so far can't get it to trigger with 5.3.0-rc7
Also cannot reproduce, 10 for 10 with 5.2.13-200.fc30.x86_64 which was just built in koji, same as 5.2.12 which was quickly replaced with a single unrelated revert for touchpads.
But comparing
fs_filllogs, yours has a ton more writer threads (~6x) and ENOSPC counts. Possibly related?
It could be related to the higher thread/ENOSPC counts, I tried to manually reproduce right now and was unsuccessful, I'll see if I can get a reproducible scenario. I'm traveling the next few weeks but will review results when I get back.
Thanks for looking into it!
Rachel