/home on BTRFS on SSD, now highly fragmenting virtuoso database - use autodefrag?
Hi!
Today I converted my /home from Ext4 to BTRFS by reformatting and copying all over again.
I created the filesystem with -l 16384 -n 16384 -d single -m single on an logical volume Intel SSD 320 and mount with compress=lzo,spacecache. Current state:
merkaba:~> btrfs filesystem show
failed to read /dev/sr0
Label: 'home' uuid: […]
Total devices 1 FS bytes used 191.98GB
devid 1 size 223.52GB used 193.02GB path /dev/dm-2
[…]
merkaba:~> btrfs filesystem df /
Data: total=14.01GB, used=9.98GB
System: total=4.00MB, used=4.00KB
Metadata: total=776.00MB, used=573.20MB
Works well so far and I see no issues.
Except for a highly fragmenting file:
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> ls -lh insgesamt 1,2G
-rw------- 1 martin martin 1,3G Jan 31 14:07 soprano-virtuoso.db
-rw-r--r-- 1 martin martin 14 Jan 31 14:06 soprano-virtuoso.lck
-rw------- 1 martin martin 0 Jan 27 17:52 soprano-virtuoso.lock
-rw-r--r-- 1 martin martin 13K Jan 31 10:22 soprano-virtuoso.log
-rw-r--r-- 1 martin martin 0 Jan 27 17:52 soprano-virtuoso.pxa
-rw-r--r-- 1 martin martin 8,0M Jan 31 14:07 soprano-virtuoso-temp.db -rw-r--r-- 1 martin martin 14K Jan 31 14:07 soprano-virtuoso.trx
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo filefrag soprano-virtuoso.db [sudo] password for martin:
soprano-virtuoso.db: 481 extents found
[…]
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo btrfs filesystem defragment soprano-virtuoso.db martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend#20> sudo filefrag soprano- virtuoso.db
soprano-virtuoso.db: 481 extents found
[…]
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sync martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo filefrag soprano-virtuoso.db soprano-virtuoso.db: 57 extents found martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo filefrag soprano-virtuoso.db [sudo] password for martin:
soprano-virtuoso.db: 3946 extents found martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo filefrag soprano-virtuoso.db soprano-virtuoso.db: 15143 extents found martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend>
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo filefrag soprano-virtuoso.db soprano-virtuoso.db: 15900 extents found
Now I do not perceive any bottle neck so far and this is an SSD.
What would be your recommendation: Add autodefrag to mount options or not?
For now I am inclined to let it run that way and see how it goes. I am a bit reluctant about autodefrag cause of the additional writes it causes.
Well, a direct dd of the file to memory is still quite fast:
merkaba:/home/martin/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> echo 3 > /proc/sys/vm/drop_caches; LANG=C dd if=soprano-virtuoso.db of=/dev/null bs=1M 1322+0 records in
1322+0 records out
1386217472 bytes (1.4 GB) copied, 5.34167 s, 260 MB/s
Another database is the Akonadi database, but I am using PostgreSQL for this at the moment which tends to use lots of (smaller) files and due to KDE PIM from KDE SC 4.4.11 still in Debian, there is no mail metadata in it, just contacts and calendar metadata.
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 1 line · Martin Steigerwald
Works well so far and I see no issues.
A fragmented ssd doesn't come with the penalty of head seek latency or rotational latency, both of which increase with fragmentation on rotating media. I'm not sure what the equivalent is for SSD. Different SSD's are doing many different things behind the scene; even when not read or written to by the file system. This is one reason why TRIM is questionable. It helps for some devices and workloads, and not others.
If you check mount, if btrfs detected from the kernel that this is non-rotating media, it should have mounted it with the ssd option. The other mount option possible, which is manually set (and unsets ssd) is ssd_spread.
Chris Murphy
Quoted 7 lines · Martin Steigerwald
Hi!
Today I converted my /home from Ext4 to BTRFS by reformatting and copying all over again.
I created the filesystem with -l 16384 -n 16384 -d single -m single on an logical volume Intel SSD 320 and mount with compress=lzo,spacecache. Current state:
The way that compression is implemented in btrfs, compressed files
always look fragmented, even if they're not. Basically, each compression block shows up as a single fragment. If the blocks are next to each other, the file's not actually fragmented, but still looks like it to fiemap.
[snip]
Quoted 13 lines
Except for a highly fragmenting file:
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> ls -lh insgesamt 1,2G
(quoted diff) +0
-7-rw------- 1 martin martin 1,3G Jan 31 14:07 soprano-virtuoso.db -rw-r--r-- 1 martin martin 14 Jan 31 14:06 soprano-virtuoso.lck -rw------- 1 martin martin 0 Jan 27 17:52 soprano-virtuoso.lock -rw-r--r-- 1 martin martin 13K Jan 31 10:22 soprano-virtuoso.log -rw-r--r-- 1 martin martin 0 Jan 27 17:52 soprano-virtuoso.pxa -rw-r--r-- 1 martin martin 8,0M Jan 31 14:07 soprano-virtuoso-temp.db -rw-r--r-- 1 martin martin 14K Jan 31 14:07 soprano-virtuoso.trx
Quoted 8 lines
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo filefrag soprano-virtuoso.db [sudo] password for martin:
soprano-virtuoso.db: 481 extents found
[…]Now I do not perceive any bottle neck so far and this is an SSD.
What would be your recommendation: Add autodefrag to mount options or not?
For an SSD, probably not worth it.
Quoted 13 lines
For now I am inclined to let it run that way and see how it goes. I am a bit reluctant about autodefrag cause of the additional writes it causes.
Well, a direct dd of the file to memory is still quite fast:
merkaba:/home/martin/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> echo 3 >
/proc/sys/vm/drop_caches; LANG=C dd if=soprano-virtuoso.db of=/dev/null bs=1M 1322+0 records in
1322+0 records out
1386217472 bytes (1.4 GB) copied, 5.34167 s, 260 MB/s
Quoted 4 lines
Another database is the Akonadi database, but I am using PostgreSQL for this at the moment which tends to use lots of (smaller) files and due to KDE PIM from KDE SC 4.4.11 still in Debian, there is no mail metadata in it, just contacts and calendar metadata.
Hugo.
Signature · 4 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- What do you give the man who has everything? -- Penicillin is ---
a good start... Am Donnerstag, 31. Januar 2013 schrieb Hugo Mills:
Quoted 6 lines · Martin Steigerwald
Quoted 5 lines
Hi!
Today I converted my /home from Ext4 to BTRFS by reformatting and copying all over again.
Quoted 8 lines
Quoted 2 lines
I created the filesystem with -l 16384 -n 16384 -d single -m single on an logical volume Intel SSD 320 and mount with compress=lzo,spacecache. Current state:
The way that compression is implemented in btrfs, compressed files
always look fragmented, even if they're not. Basically, each compression block shows up as a single fragment. If the blocks are next to each other, the file's not actually fragmented, but still looks like it to fiemap.
Ah, thanks. Thats interesting to know.
Quoted 19 lines
[snip]
Quoted 17 lines
Except for a highly fragmenting file:
martin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> ls -lh insgesamt 1,2G
(quoted diff) +0
-7-rw------- 1 martin martin 1,3G Jan 31 14:07 soprano-virtuoso.db -rw-r--r-- 1 martin martin 14 Jan 31 14:06 soprano-virtuoso.lck -rw------- 1 martin martin 0 Jan 27 17:52 soprano-virtuoso.lock -rw-r--r-- 1 martin martin 13K Jan 31 10:22 soprano-virtuoso.log -rw-r--r-- 1 martin martin 0 Jan 27 17:52 soprano-virtuoso.pxa -rw-r--r-- 1 martin martin 8,0M Jan 31 14:07 soprano-virtuoso-temp.db -rw-r--r-- 1 martin martin 14K Jan 31 14:07 soprano-virtuoso.trxmartin@merkaba:~/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend> sudo filefrag soprano-virtuoso.db [sudo] password for martin:
soprano-virtuoso.db: 481 extents found
[…]
Quoted 5 lines
Quoted 3 lines
Now I do not perceive any bottle neck so far and this is an SSD.
What would be your recommendation: Add autodefrag to mount options or not?
For an SSD, probably not worth it.
Thanks. I then will use what I advice attendees to my Linux Performance Analysis & Performance trainings and what XFS guys are also advicing:
Use the defaults
(unless you know otherwise).
Especially since I think the Intel SSD 320 is one of the better ones and desktop search is really snappy up to know.
Thanks,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7