Btrfs might be gradually slowing the boot process
Hello,
I recently noticed that my boot has become slower - it took around 29s, while at the beginning it was ~6s. I thought it was an issue with systemd, because it failed to properly indicate at which stage the slowdown occurred and how long it took. I rolled back to a pretty fresh root subvolume and the boot was fast again. However, after several reboots it started lagging again (10s? preposterous!). I decided to try the *clear_cache* mount option; it was only a guess, but it did speed the boot to the proper ~6s.
The question is - why? My filesystem is really small - I see no reason for a 5x boot slowdown. Is this a known issue? I will be glad to give any additional details about my fs.
Quoted 10 lines · On Friday, November 08, 2013 07:35:18 PM [email protected] wrote:
Hello,
I recently noticed that my boot has become slower - it took around 29s, while at the beginning it was ~6s. I thought it was an issue with systemd, because it failed to properly indicate at which stage the slowdown occurred and how long it took. I rolled back to a pretty fresh root subvolume and the boot was fast again. However, after several reboots it started lagging again (10s? preposterous!). I decided to try the *clear_cache* mount option; it was only a guess, but it did speed the boot to the proper ~6s.
Quoted 3 lines
The question is - why? My filesystem is really small - I see no reason for a 5x boot slowdown. Is this a known issue? I will be glad to give any additional details about my fs.
Have you tried defragmenting everything, including directory objects? despite of small amount of data, it could be massively fragmented, slowing down everything.
HTH
Signature · 2 lines
Regards Shridhar
Quoted 9 lines · [email protected] <[email protected]> schrieb:
I recently noticed that my boot has become slower - it took around 29s, while at the beginning it was ~6s. I thought it was an issue with systemd, because it failed to properly indicate at which stage the slowdown occurred and how long it took. I rolled back to a pretty fresh root subvolume and the boot was fast again. However, after several reboots it started lagging again (10s? preposterous!). I decided to try the *clear_cache* mount option; it was only a guess, but it did speed the boot to the proper ~6s.
Quoted 3 lines
The question is - why? My filesystem is really small - I see no reason for a 5x boot slowdown. Is this a known issue? I will be glad to give any additional details about my fs.
I'm using a similar setup (although my btrfs rootfs is a 3 disk array and is more or less huge) and I didn't see such a vast boot slowdown. Did you check that systemd is actually using the readahead services which will relocate and defragment your boot files automatically?
HTH
Kai
What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.
Chris Murphy
Chris Murphy posted on Sat, 09 Nov 2013 21:53:10 -0700 as excerpted:
Quoted 2 lines
What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.
I saw the discussion of that, but two questions it didn't answer -- (1) what was the default size before, and (2) how do I check what I actually have?
The wiki mkfs.btrfs page answers the first question: 4KiB, standard page- size. (Of course that needs updated now.)
The same page says nodesize defaults to the same as leafsize. Is that still true, or does it remain 4KiB? Also, sectorsize defaults to 4KiB, but the wiki doesn't note whether it always defaults to that, or whether it too defaults to the same as leafsize.
And nowhere on the wiki do I see a sysadmin's level guide to how leafsize, nodesize and sectorsize relate to each other, nor how they relate to chunks or how btrfs manages chunksize.
Josef Bacik's article, "Btrfs - The Swiss Army Knife of Storage"[1], linked below and in the wiki articles section, does mention that nodes and leaves are b-tree terms, where nodes contain keys and links to the next level nodes or leaves, and leaves contain actual data, which helps somewhat. But where do sectors fit in to this?
The same article says data chunk sizes are 1GiB by default, metadata 256MiB, but when space starts getting tight (and how tight is tight, with the last full gig be allocated to a full gig data chunk or will it go smaller before it reserves the last fill gig? how much before, 10 gig, 2 gig?), does it shrink by orders of two or does it jump down from a gig to say 128 meg in one jump, and do data and metadata follow the same jump rules or not?
And hardly any of this is actually on the wiki, at least the sysadmin's docs (the dev docs might cover it, I'm not a dev and haven't looked that closely at them, but that still leaves sysadmins).
[1] http://static.usenix.org/publications/login/2012-02/openpdfs/Bacik.pdf
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 7 lines · Duncan
Chris Murphy posted on Sat, 09 Nov 2013 21:53:10 -0700 as excerpted:
Quoted 2 lines
What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.
I saw the discussion of that, but two questions it didn't answer -- (1) what was the default size before, and (2) how do I check what I actually have?
I've attached the Debian patch to the magic database for file(1) to display BTRFS data, it gives output such as the following. The magic database displayed the leafsize before this patch, but the patch displays the UUID (which you REALLY want to know) and all the other information I could figure out.
# file -s /dev/dm-0 /dev/dm-0: BTRFS Filesystem sectorsize 4096, nodesize 4096, leafsize 4096, UUID=586e6f48-2985-4115-9f89-f844b319c7c0, 82545332224/115536781312 bytes used, 1 devices
I might file another bug report with a new version to display error counts etc. I think that all data which we can extract should be displayed to the user. As an aside, I don't know if the bytes used field is going to be very useful on a RAID-0 filesystem, but it's there and I think we might as well show it.
Signature · 2 lines
My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Author: Russell Coker <[email protected]>
Description: Adding updated magics for BTRFS (Closes: #663454).
magic/Magdir/filesystems +15 -6
@@ -1894,12 +1894,21 @@ >>0x10060 string >\0 lockproto %s) # BTRFS -0x10040 string _BHRfS_M BTRFS Filesystem ->0x1012b string >\0 (label "%s", ->0x10090 lelong x sectorsize %d, ->0x10094 lelong x nodesize %d, ->0x10098 lelong x leafsize %d) - +# Russell Coker <[email protected]> +0x10040 string _BHRfS_M BTRFS Filesystem +>0x1012b string >\0 label "%s", +>0x10090 lelong x sectorsize %d, +>0x10094 lelong x nodesize %d, +>0x10098 lelong x leafsize %d, +>0x10020 belong x UUID=%8x- +>0x10024 beshort x \b%4x- +>0x10026 beshort x \b%4x- +>0x10028 beshort x \b%4x- +>0x1002a beshort x \b%4x +>0x1002c belong x \b%8x, +>0x10078 lequad x %lld/ +>0x10070 lequad x \b%lld bytes used, +>0x10088 lequad x %lld devices # dvdisaster's .ecc # From: "Nelson A. de Oliveira" <[email protected]>
Hi!
Quoted 2 lines
What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.
Since my btrfs is about 2 years old I suppose I'm still using 4kB leafsize. Is there a way to change it without recreating the whole filesystem from scratch?
Regards,
Kai
Quoted 8 lines · Kai Krakow
Hi!
Quoted 2 lines
What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.
Since my btrfs is about 2 years old I suppose I'm still using 4kB leafsize. Is there a way to change it without recreating the whole filesystem from scratch?
No, it's one of the few options that's not changable after mkfs.
Hugo.
Signature · 4 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- If the first-ever performance is the première, is the ---
last-ever performance the derrière?