Skip to contentSkip to conversation

Btrfs might be gradually slowing the boot process

DISCUSSION [email protected] 7 participants 8 messages Nov 8, 2013 – Nov 10, 2013 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hello,

I recently noticed that my boot has become slower - it took around 29s, while at the beginning it was ~6s. I thought it was an issue with systemd, because it failed to properly indicate at which stage the slowdown occurred and how long it took. I rolled back to a pretty fresh root subvolume and the boot was fast again. However, after several reboots it started lagging again (10s? preposterous!). I decided to try the *clear_cache* mount option; it was only a guess, but it did speed the boot to the proper ~6s.

The question is - why? My filesystem is really small - I see no reason for a 5x boot slowdown. Is this a known issue? I will be glad to give any additional details about my fs.

#1 lore raw
Quoted 10 lines · On Friday, November 08, 2013 07:35:18 PM [email protected] wrote:

Hello,

I recently noticed that my boot has become slower - it took around 29s, while at the beginning it was ~6s. I thought it was an issue with systemd, because it failed to properly indicate at which stage the slowdown occurred and how long it took. I rolled back to a pretty fresh root subvolume and the boot was fast again. However, after several reboots it started lagging again (10s? preposterous!). I decided to try the *clear_cache* mount option; it was only a guess, but it did speed the boot to the proper ~6s.

Quoted 3 lines

The question is - why? My filesystem is really small - I see no reason for a 5x boot slowdown. Is this a known issue? I will be glad to give any additional details about my fs.

Have you tried defragmenting everything, including directory objects? despite of small amount of data, it could be massively fragmented, slowing down everything.

HTH

Signature · 2 lines
Regards
 Shridhar
#2 lore raw
In reply to [email protected]
Quoted 9 lines · [email protected] <[email protected]> schrieb:

I recently noticed that my boot has become slower - it took around 29s, while at the beginning it was ~6s. I thought it was an issue with systemd, because it failed to properly indicate at which stage the slowdown occurred and how long it took. I rolled back to a pretty fresh root subvolume and the boot was fast again. However, after several reboots it started lagging again (10s? preposterous!). I decided to try the *clear_cache* mount option; it was only a guess, but it did speed the boot to the proper ~6s.

Quoted 3 lines

The question is - why? My filesystem is really small - I see no reason for a 5x boot slowdown. Is this a known issue? I will be glad to give any additional details about my fs.

I'm using a similar setup (although my btrfs rootfs is a 3 disk array and is more or less huge) and I didn't see such a vast boot slowdown. Did you check that systemd is actually using the readahead services which will relocate and defragment your boot files automatically?

HTH
Kai

#3 lore raw

What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.

Chris Murphy

#4 lore raw

Chris Murphy posted on Sat, 09 Nov 2013 21:53:10 -0700 as excerpted:

Quoted 2 lines

What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.

I saw the discussion of that, but two questions it didn't answer -- (1) what was the default size before, and (2) how do I check what I actually have?

The wiki mkfs.btrfs page answers the first question: 4KiB, standard page- size. (Of course that needs updated now.)

The same page says nodesize defaults to the same as leafsize. Is that still true, or does it remain 4KiB? Also, sectorsize defaults to 4KiB, but the wiki doesn't note whether it always defaults to that, or whether it too defaults to the same as leafsize.

And nowhere on the wiki do I see a sysadmin's level guide to how leafsize, nodesize and sectorsize relate to each other, nor how they relate to chunks or how btrfs manages chunksize.

Josef Bacik's article, "Btrfs - The Swiss Army Knife of Storage"[1], linked below and in the wiki articles section, does mention that nodes and leaves are b-tree terms, where nodes contain keys and links to the next level nodes or leaves, and leaves contain actual data, which helps somewhat. But where do sectors fit in to this?

The same article says data chunk sizes are 1GiB by default, metadata 256MiB, but when space starts getting tight (and how tight is tight, with the last full gig be allocated to a full gig data chunk or will it go smaller before it reserves the last fill gig? how much before, 10 gig, 2 gig?), does it shrink by orders of two or does it jump down from a gig to say 128 meg in one jump, and do data and metadata follow the same jump rules or not?

And hardly any of this is actually on the wiki, at least the sysadmin's docs (the dev docs might cover it, I'm not a dev and haven't looked that closely at them, but that still leaves sysadmins).

[1] http://static.usenix.org/publications/login/2012-02/openpdfs/Bacik.pdf

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman
#5 lore raw
Quoted 7 lines · Duncan

Chris Murphy posted on Sat, 09 Nov 2013 21:53:10 -0700 as excerpted:

Quoted 2 lines

What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.

I saw the discussion of that, but two questions it didn't answer -- (1) what was the default size before, and (2) how do I check what I actually have?

I've attached the Debian patch to the magic database for file(1) to display BTRFS data, it gives output such as the following. The magic database displayed the leafsize before this patch, but the patch displays the UUID (which you REALLY want to know) and all the other information I could figure out.

# file -s /dev/dm-0
/dev/dm-0: BTRFS Filesystem sectorsize 4096, nodesize 4096, leafsize 4096, 
UUID=586e6f48-2985-4115-9f89-f844b319c7c0, 82545332224/115536781312 bytes 
used, 1 devices

I might file another bug report with a new version to display error counts etc. I think that all data which we can extract should be displayed to the user. As an aside, I don't know if the bytes used field is going to be very useful on a RAID-0 filesystem, but it's there and I think we might as well show it.

Signature · 2 lines
My Main Blog         http://etbe.coker.com.au/
My Documents Blog    http://doc.coker.com.au/

Author: Russell Coker <[email protected]>
Description: Adding updated magics for BTRFS (Closes: #663454).

magic/Magdir/filesystems +15 -6
diff -Naurp file.orig/magic/Magdir/filesystems file/magic/Magdir/filesystems
@@ -1894,12 +1894,21 @@
 >>0x10060        string          >\0             lockproto %s)
 
 # BTRFS
-0x10040         string          _BHRfS_M        BTRFS Filesystem
->0x1012b        string          >\0             (label "%s",
->0x10090        lelong          x               sectorsize %d,
->0x10094        lelong          x               nodesize %d,
->0x10098        lelong          x               leafsize %d)
-
+# Russell Coker <[email protected]>
+0x10040	string	_BHRfS_M	BTRFS Filesystem
+>0x1012b	string	>\0		label "%s",
+>0x10090	lelong	x		sectorsize %d,
+>0x10094	lelong	x		nodesize %d,
+>0x10098	lelong	x		leafsize %d,
+>0x10020	belong	x		UUID=%8x-
+>0x10024	beshort	x		\b%4x-
+>0x10026	beshort	x		\b%4x-
+>0x10028	beshort	x		\b%4x-
+>0x1002a	beshort	x		\b%4x
+>0x1002c	belong	x		\b%8x,
+>0x10078	lequad	x		%lld/
+>0x10070	lequad	x		\b%lld bytes used,
+>0x10088	lequad	x		%lld devices
 
 # dvdisaster's .ecc
 # From: "Nelson A. de Oliveira" <[email protected]>
#6 lore raw
In reply to Chris Murphy

Hi!

Quoted 2 lines

What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.

Since my btrfs is about 2 years old I suppose I'm still using 4kB leafsize. Is there a way to change it without recreating the whole filesystem from scratch?

Regards,
Kai

#7 lore raw
Quoted 8 lines · Kai Krakow

Hi!

Quoted 2 lines

What's leafsize was used when making the file system? The default is now (as of yesterday) 16KB to avoid metadata fragmentation.

Since my btrfs is about 2 years old I suppose I'm still using 4kB leafsize. Is there a way to change it without recreating the whole filesystem from scratch?

No, it's one of the few options that's not changable after mkfs.

Hugo.

Signature · 4 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- If the first-ever performance is the première,  is the ---     
                  last-ever performance the derrière?