Skip to contentSkip to conversation

bcache: ver 3

RFC PATCH Egon Alter 2 participants 2 messages Apr 24, 2010

lore
Share & tools
lore

Egon Alter

author
#0 lore raw

I'm wondering if this is similar to the "Cleancache" patches send a few days before.

Could bcache provide a (block-dev) backend to cleancache? Also there is fs- cache (for networks-fs only currently). Can these caches be combined somehow?

Egon

Kent Overstreet:

Quoted 49 lines

The core code is getting close to stable and feature complete. Garbage collection is done, and locking is mostly done (I'm not refcounting data buckets yet, and it does need to be stared at a whole lot more but it passes lockdep and seems to be working).

Basic bucket priority calculation is done (right now it acts as an LRU); you can fill the cache up and everything works like it ought to. And, I've yet to see it return bad data; I think it's not all that far off from being useful.

The one thing that has me at a loss right now is the performance isn't where it should be, and for no obvious reason. The problem is most succinctly demonstrated thusly:

With cache, no cache misses:

root@utumno:~# mount -o ro /dev/sdb /mnt/
root@utumno:~# cd /mnt/
root@utumno:/mnt# time find > /dev/null

real 0m29.618s
user 0m0.000s
sys 0m0.840s

No cache device:

root@utumno:~# mount -o ro /dev/sdb /mnt/
root@utumno:~# cd /mnt/
root@utumno:/mnt# time find > /dev/null

real 0m1.597s
user 0m0.050s
sys 0m1.240s

This is a vm, so the wall clock time in the latter case doesn't mean much, it's all in the hosts's cache. But the former case is bewildering; the cache device lives on a tmpfs so it shouldn't be waiting on IO, and cpu time is less than the uncached case. Clearly it's waiting on something, but I haven't been able to figure out what the heck to trace; there's no lock contention, blktrace shows nothing unexpected, I keep running into dead ends. Any pointers here would be much appreciated.

Besides that, there shouldn't be anything complex left before I add write support, and that should be pretty straightforward too.

This patch does have some changes to the generic bio code for completion of split bios, I'm going to split it out and submit it seperately after I've gone over it more and it's actually been tested.

Any comments/review much appreciated.
[patch sniped]

#1 lore raw
Quoted 8 lines · Egon Alter

I'm wondering if this is similar to the "Cleancache" patches send a few days before.

Could bcache provide a (block-dev) backend to cleancache? Also there is fs- cache (for networks-fs only currently). Can these caches be combined somehow?

Egon

I just looked at cleancache, and it looks like it's trying to solve part of the same problem at a different level - sort of giving the page cache a second level, instead of transparently making block devices faster. I didn't see any code for whatever holds pages when they're not in memory, I gather this was originally intended for VMs where pages in the l2 cache are still in memory, just not memory owned by the vm kernel.

Managing the cache itself is much simpler in that case; IMO most of the complexity involved in using an SSD as cache is the indexing, allocation, etc. - it's kind of like writing a filesystem, minus the notion of files, plus we can play fast and loose in certain ways.

That said, adding things to the cache should definitely be done more like the way cleancache does it; the page cache is what knows what's going on. So I don't think there's much if any code that could be shared, but definitely the hooks cleancache/ramzswap are using is what bcache should be using.

fs-cache is completely different, I haven't looked at it too much but just knowing its approach I don't think there's any way it'll be as efficient as bcache will be. It just makes more sense - makes things a lot simpler and there's less overhead involved in working with ranges of sectors than doing it at the filesystem level... and this way the allocation is determined entirely by bcache, and can be better optimized for ssd erase blocks. (Having the code run under generic_make_request does make the code more interesting since we can't sleep, but that's a solved issue and doesn't add to the work it does in the fast path).