Skip to contentSkip to conversation

dedicated metadata drives?

DISCUSSION Tomasz Chmielewski 3 participants 3 messages Jan 4, 2019 – Jan 9, 2019 Btrfs file system

lore
Share & tools
lore
#0 lore raw

According to btrfs wiki, some patches have been submitted to support metadata on different devices than data (i.e. metadata on SSD, data on HDD):

https://btrfs.wiki.kernel.org/index.php/Project_ideas#Dedicated_metadata_drives

Dedicated metadata drives

Not claimed — submitted — Not in kernel yet

We're able to split data and metadata IO very easily. Metadata tends
to be dominated by seeks and for many applications it makes sense to put the metadata onto faster SSDs.

This article (almost 2.5 years old) claims one company is already using either these patches or something similar:

https://lwn.net/Articles/698090/

August 24, 2016

To combat that, he has a set of patches to automatically put the
Btrfs metadata on SSDs. The block layer provides information on whether the storage is rotational; for now, his patch assumes that if
it is not rotational then it is fast. The patch has made a huge
difference in the latencies and requires less flash storage (e.g. 450GB for 40TB filesystem) for Facebook's file workload that
consists of a wide variety of file sizes.

Do these patches exist anywhere? I couldn't find them in the list archive.

Tomasz

#1 lore raw
Quoted 43 lines

-----Original Message-----
From: [email protected] <linux-btrfs- [email protected]> On Behalf Of Tomasz Chmielewski
Sent: Saturday, 5 January 2019 12:25 AM
To: Btrfs BTRFS <[email protected]>
Subject: dedicated metadata drives?

According to btrfs wiki, some patches have been submitted to support metadata on different devices than data (i.e. metadata on SSD, data on HDD):

https://btrfs.wiki.kernel.org/index.php/Project_ideas#Dedicated_metadata
_drives

Dedicated metadata drives

Not claimed — submitted — Not in kernel yet

We're able to split data and metadata IO very easily. Metadata tends to be
dominated by seeks and for many applications it makes sense to put the metadata onto faster SSDs.

This article (almost 2.5 years old) claims one company is already using either these patches or something similar:

https://lwn.net/Articles/698090/

August 24, 2016

To combat that, he has a set of patches to automatically put the Btrfs
metadata on SSDs. The block layer provides information on whether the storage is rotational; for now, his patch assumes that if
it is not rotational then it is fast. The patch has made a huge difference in
the latencies and requires less flash storage (e.g. 450GB for 40TB filesystem) for Facebook's file workload that
consists of a wide variety of file sizes.

Quoted 1 line

Do these patches exist anywhere? I couldn't find them in the list archive.

I managed to find this suggestion about how it could be done:
https://patchwork.kernel.org/patch/9556973/

I would love to use this feature as well. My current setup uses btrfs on lvm with caching enabled, but I would prefer a simpler setup.

Paul.

#2 lore raw
Quoted 44 lines · Paul Jones
Quoted 44 lines

-----Original Message-----
From: [email protected] <linux-btrfs- [email protected]> On Behalf Of Tomasz Chmielewski
Sent: Saturday, 5 January 2019 12:25 AM
To: Btrfs BTRFS <[email protected]>
Subject: dedicated metadata drives?

According to btrfs wiki, some patches have been submitted to support metadata on different devices than data (i.e. metadata on SSD, data on HDD):

https://btrfs.wiki.kernel.org/index.php/Project_ideas#Dedicated_metadata
_drives

Dedicated metadata drives

Not claimed — submitted — Not in kernel yet

We're able to split data and metadata IO very easily. Metadata tends to be
dominated by seeks and for many applications it makes sense to put the metadata onto faster SSDs.

This article (almost 2.5 years old) claims one company is already using either these patches or something similar:

https://lwn.net/Articles/698090/

August 24, 2016

To combat that, he has a set of patches to automatically put the Btrfs
metadata on SSDs. The block layer provides information on whether the storage is rotational; for now, his patch assumes that if
it is not rotational then it is fast. The patch has made a huge difference in
the latencies and requires less flash storage (e.g. 450GB for 40TB filesystem) for Facebook's file workload that
consists of a wide variety of file sizes.

Quoted 8 lines
Quoted 1 line

Do these patches exist anywhere? I couldn't find them in the list archive.

I managed to find this suggestion about how it could be done:
https://patchwork.kernel.org/patch/9556973/

I would love to use this feature as well. My current setup uses btrfs on lvm with caching enabled, but I would prefer a simpler setup.

Paul.

Another out of tree option are the patches from Timofey Titovets "Btrfs: enhance raid1/10 balance heuristic"

https://lore.kernel.org/linux-btrfs/[email protected]/

Then you'd put all your data drives in a single LVM volume, could be MD raid or what not, and then your SSDs in another lvol. Make your data single, and your metadata RAID1 and then the SSDs should be preferentially used for metadata access. I haven't actually tried the patches yet myself, but have a test system with the proper hardware setup now running normal btrfs 4.9. Just not enough time in a day to test everything. The SSD's do make a huge difference for metadata heavy things like running compsize on an fs.