Updated package for Fedora

Hi!

I updated the Fedora 20 & rawhide package for bcache-tools:

https://admin.fedoraproject.org/updates/bcache-tools-0-0.11.20130909git.fc20

It's an update to the latest git / gist sources for bcache-tools an bcache-status. Some of the patches it includes to make it run on Fedora are explained below.

Rolf

bcache-tools-20130827-udevfix.patch
=======================
On Fedora only 'early' udev rules files run /usr/sbin/blkid and the rest of the rules files use the results. This patch removes the call to /usr/sbin/blkid and (more important) makes sure that probe-bcache does not print overlapping variable names. This is needed until util-linux itself supports bcache, after that there's no need to call probe-bcaches at all.

@@ -5,13 +5,12 @@
 ACTION=="remove", GOTO="bcache_end"
 
 # Backing devices: scan, symlink, register
-IMPORT{program}="/sbin/blkid -o udev $tempnode"
 # blkid and probe-bcache can disagree, in which case don't register
 ENV{ID_FS_TYPE}=="?*", ENV{ID_FS_TYPE}!="bcache", 

GOTO="bcache_backing_end"

IMPORT{program}="/sbin/probe-bcache -o udev $tempnode"
-ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}" -SUBSYSTEM=="block", ACTION=="add|change", ENV{ID_FS_TYPE}=="bcache", \
+ENV{BCACHE_ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{BCACHE_ID_FS_UUID_ENC}"

+SUBSYSTEM=="block", ACTION=="add|change", ENV{BCACHE_ID_FS_TYPE}=="bcache", \
         RUN+="bcache-register $tempnode"

LABEL="bcache_backing_end"

@@ -61,9 +61,9 @@
         uuid_unparse(sb.uuid, uuid);
 
         if (udev)
-            printf("ID_FS_UUID=%s\n"
-                   "ID_FS_UUID_ENC=%s\n"
-                   "ID_FS_TYPE=bcache\n",
+            printf("BCACHE_ID_FS_UUID=%s\n"
+                   "BCACHE_ID_FS_UUID_ENC=%s\n"
+                   "BCACHE_ID_FS_TYPE=bcache\n",
                    uuid, uuid);
         else
             printf("%s: UUID=\"\" TYPE=\"bcache\"\n", uuid);

bcache-tools-20130827-register.patch
=======================
Fedora needs full path names to call /sbin/ binaries from udev, this is fixed in this patch. Also errors are suppressed when a device is registered twice.

@@ -1,4 +1,9 @@
 #!/bin/sh
-modprobe -qba bcache
-test -f /sys/fs/bcache/register_quiet && echo "$1" > /sys/fs/bcache/register_quiet
+#
+# Utility script to be called from udev, so it's not monkey proof at all. It's
+# sole purpose is to load the bcache kernel module whenever a bcache device is
+# detected.
+#
+/sbin/modprobe -qba bcache
+test -f /sys/fs/bcache/register_quiet && echo "$1" > /sys/fs/bcache/register_quiet 2>/dev/null

Hi,

I'd like to report some other nice progress as well, related to other Linux packages. Although I'm working on Fedora packaging, there is good spin-off for all Linux distro's out there!

Util-Linux
=====
Util-Linux 2.24 will have support for bcache built in (for blkid and wipefs): https://bugzilla.redhat.com/show_bug.cgi?id=1001120. This means that "stuff gets more straightforward", e.g. the udev rules for bcache can simply rely on bcache instead of also having to call probe-bcache. There's also benefits for Dracut, see below.

Dracut
====
Dracut 032 already had bcache support, dracut 033 now has support built in for the migration phase from probe-bcache to blkid. I did some testing with a root FS on bcache, and dracut perfectly builds a booting initramfs! Beware though that there is a (soft) dependency on Util-Linux 2.24. When using an older Util-linux users should pass the -N option to dracut te build a functional initramfs. More details: https://bugzilla.redhat.com/show_bug.cgi?id=1003207

I'm aware that other distro's don't use dracut to build an initramfs, but the dracut related info may be useful."

LVM2
===
LVM2 needs a small change to be able to detect PV's on bcache devices. Without the change users need to add a "types = [ "bcache", 1 ]" line in the /etc/lvm/lvm.conf. Users may also need to pass the --lvmconf option to dracut when booting from a root FS on bcache. More details: https://bugzilla.redhat.com/show_bug.cgi?id=1000817

In the bugzilla there's a claim that bcache support in Fedora isn't supportable because upstream (bcache) is'nt actively working with Fedora. I don't agree with this, because I think the collaboration with upstream is working fine.

I'll keep you all posted,

Rolf

Hi,

Like I did on the 14th on the linux-bcache list I'd like to send an update on the progress of bcache related packages. While focussing on Fedora packaging of bcache-tools, I had some good collaboration with other packagers resulting in improved bcache support in other packages as well. Other Linux distro's may benefit from these updated packages too.

util-linux
==========
On 27th of September util-linux v2.24 RC will probably be released. This release supports the identification of bcache superblocks in libblkid, actually integrating and obsoleting probe-bcache. Because of this udev rules for bcache can be simplified because they only need blkid. Of course this only applies to systems where util-linux v2.24 RC (or higher) actually is installed, otherwise probe-bcache is still needed.

Dracut
======
Since version 032 Dracut already had bcache support. Since version 033 it adds support for util-linux v2.24 RC, so even without (obsoleted) probe-bcache it is able to identify bcache using blkid. Additionally Dracut benefits from blkid being able to identify bcache, because it uses blkid to identify all necessary kernel modules automatically, so no -N option is needed anymore.

LVM2
====
With the release of v2_01_102 LVM2 now accepts Physical Volumes on bcache by default. This simplifies the creation of initramfs (by Dracut) because no specific LVM2 option (--lvmconf) needs to be passed to Dracut.For the user this means that updating a kernel just works out of the box!

bcache-tools
============
The bcache-tools package is available in Fedora 20. I plan to build an updated package that no longer includes probe-bcache when the new util-linux is released.

Anaconda
========
Anaconda (the Fedora installer) does not support bcache yet. This is planned for Fedora 20. This is important when installing Fedora on a system and having your root filesystem on bcache. Althought other Distro's don't use Anaconda, I guess their installers also need to be changed in some way to supportbcache.

Currently having Fedora installed with your root Filesystem on bcache is possible, but It's done in a fewsteps:
1. Install Fedora using Anaconda without using bcache, but create an extra partition to supportan alternate root FS
2. From the running system build a bcache device using the extra partition. Copy the current root FS to the bcache root FS
3. Reboot your system in the bcache root FS, and reclaim the spaces used by the non-bcache root partition

More information can be found below, related to the "SSD Cache Fedora test day".

The SSD Cache Fedora test day
=============================
On 13th of October there's an "SSD Cache Fedora test day": see the Wiki page https://fedoraproject.org/wiki/Test_Day:2013-10-13_SSD_Cache. This page is work in progress, any feedback is welcome. People interested in testing are invited to participate on 13th of October.

When there's anything new toreport, I'll keep you posted.

Rolf Fokkens

Hi

It was brought to my attention that there's a confusing typo in the message below regarding the planning of bcache support for anaconda. The claim that it is planned for F20 is not true, it is planned for F21:

https://fedorahosted.org/fesco/ticket/1145

Sorry for any confusion,

Rolf

Op 26-09-13 11:56 schreef Rolf Fokkens <[email protected]>:

Is there no chance for Intel SRT support in the kernel? There's an increasing amount of hardware shipping with the feature enabled (e.g. from HP) and of course this isn't supported. Presently anaconda indicates to the user there are no drives at all attached, no destination for installation.

Chris Murphy

devel mailing list
[email protected]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

On Thu, Sep 26, 2013 at 11:56:25AM +0200, Rolf Fokkens wrote:
[...]

Hi Rolf,

could you spend few words about the results
of the SSD Cache Fedora test day?

Anything interesting or surprising happened?
Any disappointment?

Thanks,

bye,

piergiorgio

Hi all,

We've been waiting to see if other people who couldn't make on Sunday would also do some testing. That hasn't happened, so a few words on my impressions on the test day are appropriate indeed.

First of all not many people really did some testing. We didn't expect many people to participate, but the 3 people who did (many thanks to them!) were the bare minimum we anticipated. This was probably caused by the following:
- SSD caching may need more explanation, not many people understand what it is and what the benefits are
- Because it's hard to change an existing partition to a 'bcached' partition, it's not really tempting to test (there's a blocks utility under development that may help, currently backup-restore is the only way).
- Not many people have the required resources available to do testing. Even when testing in a VM not many people have the required 10GB available (The requirements could be lowered top about 6GB, so that might help)
- Installing F20 as requested in the prerequisites was harder to the testers than we anticipated. Specifically planning a specific partition layout in Anaconda requires a lot of attention (I could upload a VM image somewhere to facilitate that).

About the testing itself:
- the alignment of the tools (bcache-tools, kernel, util-linux and dracut) is really good now, people were able to do the testcases (1.A and 1.B) without a hitch.
- nobody tested the LVM integration (testcases 2.A and 2.B), so no test results on that part.
- Unfortunately kernel 3.11.4 (which was the latest version on Sunday) exhibited a bug during stress testing
(https://bugzilla.redhat.com/show_bug.cgi?id=1018615), but that bug is supposed to be fixed in kernel 3.11.5 which was released later this week.

So I think SSD Caching (using bcache) is in a good shape, but I would like to encourage people to do some more testing. Of course other feedback is also appreciated.

Thanks,

Rolf

Op 10/18/13 11:22 AM schreef Piergiorgio Sartor <[email protected]>:

Hi,

Good work, and thanks for the detailed summary!

Regards,

Hans

devel mailing list
[email protected]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

blocks[1] has been doing bcache conversion since forever (last march, when the project was renamed from lvmify).

Backup-restore hasn't been necessary since then, apart from unusual cases, like not having a /boot partition.

[1] https://github.com/g2p/blocks

Hi,

first of all thank to Rolf for the report.
It helps a lot to understand the current
status and what it needs to be done.
Thanks again!

I've got a question about this.

I've two HDDs combined with "md" RAID-10 and, on top of the RAID, LVM volumes (there is a separted RAID-1 for /boot, too).

Does the tools converts the complete RAID-10, including the LVM volumes to bcache?

Some times ago I asked, in my setup, what would be the right approach to bcache, between "caching" the RAID or caching each LVM volume and the answer was that the usual way is to cache the RAID.

Any changes on this statement?

Thanks,

bye,

pg

piergiorgio

Op 10/20/13 8:59 PM schreef Piergiorgio Sartor <[email protected]>:

I will look into the blocks tool for F21, but for now I leave answering the question to Gabriel.

I can imagine that one may wish to (b)cache one specific LV and not the other(s) in the same VG in which case bcache is to be used on top of the specific LV. I think in general however it's best to use LVM on top of bcache because it's closer to the (slow) storage you want to accelerate. So I guess it depends on your requirements.

I'm not sure if there are specific technical arguments in favour of one or the other. But I can say I've seen both work during testing.

Rolf

On Mon, Oct 21, 2013 at 02:34:35PM +0200, Rolf Fokkens wrote:
[...]

Interesting, then logic would suggest to (b)cache the components of the RAID.
Of, even better, to (b)cache /dev/sd[ab] (in this case) and cover, in this way, everything.

It's a desktop PC, I notice that performances mainly depend on the storage subystem, the rest (CPU, memory, GPU) is fine for me.

Thanks,

bye,

piergiorgio

Well, I agree, there's more to it. Like cost. One could consider to pair serveral HHD' each with a dedicated bcache SSD. And from that one could build a RAID array. This RAID array has excellent (read) performance because of the combined SSD performance. This storage system does break when one of the HDD's or SDD's breaks, which is also a nice feature. So if it weren't for the cost (and the number of availbale SATA connectors) this could be interresting. But it's all a matter of requirements of course.

In my desktop PC I'm running singls SSD bcache on a RAID5 array. The performance is fine, and when the SSD breaks, well, I hope the FS on the HDD can be recovered. And if not? Well, it's only a desktop PC.

And when using multiple SSD's for reduncancy, the following article covers some interresting features in bcache:

http://www.linux.com/news/featured-blogs/200-libby-clark/728209-about-the-linux-kernel-bcache

Quote "Multiple SSDs will allow us to mirror dirty data and metadata, but not clean data - you get redundancy without wasting SSD space duplicating clean cached data".

devel mailing list
[email protected]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

I believe that quote describes "what we'd like to do someday", not "what it does now"?

You're right, it's about plans for the future.

I noticed a somewhat confusing typo, the right sentence is "This storage system does NOT break when one of the HDD's or SDD's breaks".

Hi,

I tested F20 on a physical host with a vertex3 ssd.
But it was not a requested test case, i re-used a md raid0 with a 6gb partition of the ssd.

The file-system was setup as a storage pool for virt-manager.

I did not experience any issues, i installed 2 F20 guest at the same time. I even rebooted many times and it survived several kernel updates. I did not test a power-loss scenario. I has been working ok since a couple of days.

The only thing that i can recall as strange is the fact that bcache seems to not to use discard/trim for some reason.

I also did the non lvm tests on guest, without any issue.

Cheers.

devel mailing list
[email protected]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Hi Reartes,

Thanks for your feedback, glad to hear it works well for you.

discard/trim can be enabled, see:
http://evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache-dev#n126

There's also an explanation in the document why it isn't enabled by default:
"|Defaults to off, since SATA TRIM is an unqueued command (and thus slow)."

Rolf
|

devel mailing list
[email protected]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

default:

Is there any way to TRIM all the empty space in bulk occasionally at off hours (like fstrim), or if you don't enable TRIM the only way to clean up would be to occasionally disable the cache, do a secure erase (or something), and then recreate/add it back?

Thanks.