Skip to contentSkip to conversation

Add dm-writeboost (log-structured caching target)

PATCH New Akira Hayakawa 1 participant 1 message Jan 11, 2014

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Add dm-writeboost (log-structured caching target)

dm-writeboost is an another cache target like dm-cache and bcache. The biggest difference from existing cache softwares is that it focuses on bursty writes.

dm-writeboost first writes the data to RAM buffer and makes a log containing both data and their metadata.
The log is written to the cache device in log-structured manner. The fact that the log contains metadata of the data blocks makes dm-writeboost is robust for power fault. It can replay the log after crash.

---
 Documentation/device-mapper/dm-writeboost.txt |  161 +++
 drivers/md/Kconfig                            |    8 +
 drivers/md/Makefile                           |    3 +
 drivers/md/dm-writeboost-daemon.c             |  520 ++++++++++
 drivers/md/dm-writeboost-daemon.h             |   40 +
 drivers/md/dm-writeboost-metadata.c           | 1352 +++++++++++++++++++++++++
 drivers/md/dm-writeboost-metadata.h           |   51 +
 drivers/md/dm-writeboost-target.c             | 1258 +++++++++++++++++++++++
 drivers/md/dm-writeboost.h                    |  464 +++++++++
 9 files changed, 3857 insertions(+)
 create mode 100644 Documentation/device-mapper/dm-writeboost.txt
 create mode 100644 drivers/md/dm-writeboost-daemon.c
 create mode 100644 drivers/md/dm-writeboost-daemon.h
 create mode 100644 drivers/md/dm-writeboost-metadata.c
 create mode 100644 drivers/md/dm-writeboost-metadata.h
 create mode 100644 drivers/md/dm-writeboost-target.c
 create mode 100644 drivers/md/dm-writeboost.h
Documentation/device-mapper/dm-writeboost.txt new file +161 -0
new file mode 100644
index 0000000..0161663
@@ -0,0 +1,161 @@
+dm-writeboost
+=============
+Writeboost target provides log-structured caching.
+It batches random writes into a big sequential write to a cache device.
+
+It is like dm-cache as a cache target but the difference is that Writeboost
+focuses on bursty writes and the lifetime of the SSD cache device.
+
+More documents and tests are available in
+https://github.com/akiradeveloper/dm-writeboost
+
+Design
+======
+There are 1 foreground and 6 background processes.
+
+Foreground
+----------
+It accepts bios and stores the write data to RAM buffer.
+When the buffer is full, it creates a "flush job" and queues it.
+
+Background
+----------
+* wbflusher (Writeboost flusher)
+Executes a flush job.
+wbflusher exploits workqueue mechanism and may run in parallel.
+It exhibits the sysfs (/sys/bus/workqueue/devices/wbflusher)
+to control the behavior.
+
+* Barrier deadline worker
+Barrier flags such as REQ_FUA and REQ_FLUSH are acked lazily.
+Immediately handling these bios badly deteriorate the throughput.
+Bios with these flags are queued and forcefully processed at worst
+within `barrier_deadline_ms` period.
+
+* Migrate Daemon
+It migrates, or writes back, cache data to backing store.
+
+If `allow_migrate` is true, it migrates without impending situation.
+Being in impending situation is that there are no room in cache device
+for writing more flush jobs.
+
+Migration is done batching `nr_max_batched_migration` segments at maximum
+at a time. Thus, unlike existing I/O scheduler, two dirty writes close in
+positional space but distant in time space can be merged. Writetboost is
+also a extension of I/O scheduler.
+
+* Migration Modulator
+Migration while the backing store is heavily loaded grows the device queue
+longer and affects the read from the backing store.
+Migration modulator surveils the load of the backing store and turns on/off
+the migration by switching `allow_migrate`.
+
+* Superblock Recorder
+Superblock is a last sector of first 1MB region in cache device containing
+what id of the segment lastly migrated. This daemon periodically updates
+the region every `update_record_interval` seconds.
+
+* Sync Daemon
+This daemon forcefully writes out all the dirty data persistently every
+`sync_interval` seconds. Some careful users want to make all the writes
+persistent periodically.
+
+Target Interface
+================
+All the operations are via dmsetup command.
+
+Constructor
+-----------
+<type>
+<essential args>*
+<#optional args> <optional args>*
+<#tunable args> <tunable args>* (see 'Message')
+
+Optionals are tunables are unordered lists of Key-Value pairs.
+
+Essential args and optional args are different for different buffer type.
+
+<type> (The type of the RAM buffer)
+0: volatile RAM buffer (DRAM)
+1: non-volatile buffer with a block I/F
+2: non-volatile buffer with PRAM I/F
+
+Currently, only type 0 is supported.
+
+Type 0
+------
+<essential args>
+backing_dev        : Slow device holding original data blocks.
+cache_dev          : Fast device holding cached data and its metadata.
+
+<optional args>
+segment_size_order : The size of RAM buffer
+                     1 << n (sectors), 4 <= n <= 10
+                     default 7
+rambuf_pool_amount : The amount of the RAM buffer pool (kB).
+                     Too fewer amount may cause waiting for new buffer
+                     to become available again. But too much doesn't
+		     benefit the performance.
+                     default 2048
+
+Note that cache device is re-formatted if the first sector of the cache
+device is zeroed out.
+
+Status
+------
+<cursor pos>
+<#cache blocks>
+<#segments>
+<current id>
+<lastly flushed id>
+<lastly migrated id>
+<#dirty cache blocks>
+<stat (w/r) x (hit/miss) x (on buffer?) x (fullsize?)>
+<#not full flushed>
+<#tunable args> [tunable args]
+
+Messages
+--------
+You can tune up the behavior of writeboost via message interface.
+
+* barrier_deadline_ms (ms)
+Default: 3
+All the bios with barrier flags like REQ_FUA or REQ_FLUSH
+are guaranteed to be acked within this deadline.
+
+* allow_migrate (bool)
+Default: 1
+Set to 1 to start migration.
+
+* enable_migration_modulator (bool) and
+  migrate_threshold (%)
+Default: 1 and 70
+Set to 1 to run migration modulator.
+Migration modulator surveils the load of backing store and sets the
+migration started if the load is lower than the `migrate_threshold`.
+
+* nr_max_batched_migration (int)
+Default: 1MB / segment size
+Number of segments to migrate at a time.
+Set higher value to fully exploit the capacily of the backing store.
+Even a single HDD is capable of processing 1MB/sec random writes so
+the default value is set to 1MB / segment size. Set higher value if
+you use RAID-ed drive as the backing store.
+
+* update_record_interval (sec)
+Default: 60
+The superblock record is updated every update_record_interval seconds.
+
+* sync_interval (sec)
+Default: 60
+All the dirty writes are guaranteed to be persistent every this interval.
+
+Example
+=======
+dmsetup create writeboost-vol --table "0 ${sz} 0 writeboost ${BACKING} {CACHE}"
+dmsetup create writeboost-vol --table "0 ${sz} 0 writeboost ${BACKING} {CACHE} \
+                                       4 rambuf_pool_amount 8192 segment_size_order 8 \
+				       2 allow_migrate 1"
+dmsetup create writeboost-vol --table "0 ${sz} 0 writeboost ${BACKING} {CACHE} \
+                                       0 \
+				       2 allow_migrate 1"
drivers/md/Kconfig +8 -0
index f2ccbc3..65a6d95 100644
@@ -290,6 +290,14 @@ config DM_CACHE_CLEANER
          A simple cache policy that writes back all data to the
          origin.  Used when decommissioning a dm-cache.
 
+config DM_WRITEBOOST
+	tristate "Log-structured Caching (EXPERIMENTAL)"
+	depends on BLK_DEV_DM
+	default y
+	---help---
+	  A cache layer that batches random writes into a big sequential
+	  write to a cache device in log-structured manner.
+
 config DM_MIRROR
        tristate "Mirror target"
        depends on BLK_DEV_DM
drivers/md/Makefile +3 -0
index 2acc43f..6db61ce 100644
@@ -14,6 +14,8 @@ dm-thin-pool-y	+= dm-thin.o dm-thin-metadata.o
 dm-cache-y	+= dm-cache-target.o dm-cache-metadata.o dm-cache-policy.o
 dm-cache-mq-y   += dm-cache-policy-mq.o
 dm-cache-cleaner-y += dm-cache-policy-cleaner.o
+dm-writeboost-y	+= dm-writeboost-target.o dm-writeboost-metadata.o \
+			dm-writeboost-daemon.o
 md-mod-y	+= md.o bitmap.o
 raid456-y	+= raid5.o
 
@@ -52,6 +54,7 @@ obj-$(CONFIG_DM_VERITY)		+= dm-verity.o
 obj-$(CONFIG_DM_CACHE)		+= dm-cache.o
 obj-$(CONFIG_DM_CACHE_MQ)	+= dm-cache-mq.o
 obj-$(CONFIG_DM_CACHE_CLEANER)	+= dm-cache-cleaner.o
+obj-$(CONFIG_DM_WRITEBOOST)	+= dm-writeboost.o
 
 ifeq ($(CONFIG_DM_UEVENT),y)
 dm-mod-objs			+= dm-uevent.o
drivers/md/dm-writeboost-daemon.c new file +520 -0

520 lines not on this page · raw message

drivers/md/dm-writeboost-daemon.h new file +40 -0
new file mode 100644
index 0000000..7e913db
@@ -0,0 +1,40 @@
+/*
+ * Copyright (C) 2012-2014 Akira Hayakawa <[email protected]>
+ *
+ * This file is released under the GPL.
+ */
+
+#ifndef DM_WRITEBOOST_DAEMON_H
+#define DM_WRITEBOOST_DAEMON_H
+
+/*----------------------------------------------------------------*/
+
+void flush_proc(struct work_struct *);
+void wait_for_flushing(struct wb_device *, u64 id);
+
+/*----------------------------------------------------------------*/
+
+void queue_barrier_io(struct wb_device *, struct bio *);
+void barrier_deadline_proc(unsigned long data);
+void flush_barrier_ios(struct work_struct *);
+
+/*----------------------------------------------------------------*/
+
+int migrate_proc(void *);
+void wait_for_migration(struct wb_device *, u64 id);
+
+/*----------------------------------------------------------------*/
+
+int modulator_proc(void *);
+
+/*----------------------------------------------------------------*/
+
+int sync_proc(void *);
+
+/*----------------------------------------------------------------*/
+
+int recorder_proc(void *);
+
+/*----------------------------------------------------------------*/
+
+#endif
drivers/md/dm-writeboost-metadata.c new file +1352 -0

1,352 lines not on this page · raw message

drivers/md/dm-writeboost-metadata.h new file +51 -0

51 lines not on this page · raw message

drivers/md/dm-writeboost-target.c new file +1258 -0

847 lines not on this page · raw message

drivers/md/dm-writeboost.h new file +464 -0

464 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.8.3.4

Part of the diff is not shown here. View the raw message.