MD: a caching layer for raid5/6
v3 cover MD: a caching layer for raid5/6
Hi,
This is the third version of the raid5/6 caching layer patches. The patches add a caching layer for raid5/6. The caching layer uses a SSD as a cache for a raid 5/6. It works like the similar way of a hardware raid controller. The purpose is to improve raid performance (reduce read-modify-write) and fix write hole issue. The main patch is patch 3 and the description has all details about the implementation. Please review!
Thanks,
Shaohua
V3:
-make reclaim multi-thread
-add statistics in sysfs
-bug fixes
V2:
-metadata write doesn't use FUA
-discard request is only issued when necessary
-bug fixes and cleanup
Shaohua Li (7):
raid5: directly use mddev->queue
raid5: A caching layer for RAID5/6
raid5: add some sysfs entries
md: don't allow resize/reshape with cache support
raid5: skip resync if caching is enabled
raid5: guarantee cache release stripes in correct way
raid5: multi-thread support for raid5 caching reclaim
Song Liu (1):
MD: add a new disk role to present cache device
drivers/md/Makefile | 2 +- drivers/md/md.c | 14 +- drivers/md/md.h | 4 + drivers/md/raid5-cache.c | 3775 ++++++++++++++++++++++++++++++++++++++++ drivers/md/raid5.c | 177 +- drivers/md/raid5.h | 25 +- include/uapi/linux/raid/md_p.h | 73 + 7 files changed, 4022 insertions(+), 48 deletions(-) create mode 100644 drivers/md/raid5-cache.c
Signature · 1 line
1.8.1
PATCH v3 1/8 MD: add a new disk role to present cache device
From: Song Liu <[email protected]>
Next patches will use a disk as raid5/6 caching. We need a new disk role to present the cache device
Not sure if we should bump up the MD superblock version for the disk role.
--- drivers/md/md.c | 14 +++++++++++++- drivers/md/md.h | 4 ++++ include/uapi/linux/raid/md_p.h | 1 + 3 files changed, 18 insertions(+), 1 deletion(-)
drivers/md/md.c +13 -1
@@ -1656,6 +1656,9 @@ static int super_1_validate(struct mddev *mddev, struct md_rdev *rdev)case 0xfffe :/* faulty */ set_bit (Faulty, &rdev->flags);break ; +case 0xfffd :/* cache device */ +set_bit (WriteCache, &rdev->flags); +break ;default : rdev->saved_raid_disk = role; if ((le32_to_cpu(sb->feature_map) &
@@ -1811,6 +1814,8 @@ static void super_1_sync(struct mddev *mddev, struct md_rdev *rdev) sb->dev_roles[i] =cpu_to_le16 (0xfffe ); else if (test_bit(In_sync, &rdev2->flags)) sb->dev_roles[i] = cpu_to_le16(rdev2->raid_disk); + else if (test_bit(WriteCache, &rdev2->flags)) + sb->dev_roles[i] = cpu_to_le16(0xfffd); else if (rdev2->raid_disk >= 0) sb->dev_roles[i] = cpu_to_le16(rdev2->raid_disk); else
@@ -5780,7 +5785,8 @@ static int get_disk_info(struct mddev *mddev, void __user * arg)else if (test_bit (In_sync, &rdev->flags)) { info.state |= (1 <<MD_DISK_ACTIVE ); info.state |= (1 <<MD_DISK_SYNC ); - } + }else if (test_bit (WriteCache, &rdev->flags)) + info.state |= (1 <<MD_DISK_WRITECACHE );if (test_bit (WriteMostly, &rdev->flags)) info.state |= (1 <<MD_DISK_WRITEMOSTLY ); }else {
@@ -5895,6 +5901,8 @@ static int add_new_disk(struct mddev *mddev, mdu_disk_info_t *info)else clear_bit (WriteMostly, &rdev->flags); +if (info->state & (1 <<MD_DISK_WRITECACHE )) +set_bit (WriteCache, &rdev->flags);/* * check whether the device shows up in other nodes */
@@ -7263,6 +7271,10 @@ static int md_seq_show(struct seq_file *seq, void *v)seq_printf (seq,"(F)" );continue ; } +if (test_bit (WriteCache, &rdev->flags)) { +seq_printf (seq,"(C)" ); +continue ; + }if (rdev->raid_disk <0 )seq_printf (seq,"(S)" );/* spare */ if (test_bit (Replacement, &rdev->flags))
drivers/md/md.h +4 -0
@@ -175,6 +175,10 @@ enum flag_bits {* This device is seen locally but not * by the whole cluster */ + WriteCache, /* This device is used as write cache. +* Usually, this device should be faster +* than other devices in the array +*/ };#define BB_LEN_MASK (0x00000000000001FFULL)
include/uapi/linux/raid/md_p.h +1 -0
@@ -89,6 +89,7 @@* read requests will only be sent here in * dire need */ +#define MD_DISK_WRITECACHE 18/* disk is used as the write cache in RAID-5/6 */ typedef struct mdp_device_descriptor_s { __u32 number; /* 0 Device number in the entire set */
Signature · 1 line
1.8.1
Re: [PATCH v3 1/8] MD: add a new disk role to present cache device
Quoted 7 lines · On Wed, 3 Jun 2015 15:48:36 -0700 Shaohua Li <[email protected]> wrote:
From: Song Liu <[email protected]>
Next patches will use a disk as raid5/6 caching. We need a new disk role to present the cache device
Not sure if we should bump up the MD superblock version for the disk role.
No need to increase the superblock version, but you would need to add a feature flag (for feature_map) which was set whenever the array had a caching device.
NeilBrown
Quoted 89 lines
Signed-off-by Song Liu <[email protected]>Signed-off-by Shaohua Li <[email protected]>--- drivers/md/md.c | 14 +++++++++++++- drivers/md/md.h | 4 ++++ include/uapi/linux/raid/md_p.h | 1 + 3 files changed, 18 insertions(+), 1 deletion(-)drivers/md/md.c +13
-1@@ -1656,6 +1656,9 @@ static int super_1_validate(struct mddev *mddev, struct md_rdev *rdev)case 0xfffe :/* faulty */ set_bit (Faulty, &rdev->flags);break ; +case 0xfffd :/* cache device */ +set_bit (WriteCache, &rdev->flags); +break ;default : rdev->saved_raid_disk = role; if ((le32_to_cpu(sb->feature_map) &@@ -1811,6 +1814,8 @@ static void super_1_sync(struct mddev *mddev, struct md_rdev *rdev) sb->dev_roles[i] =cpu_to_le16 (0xfffe ); else if (test_bit(In_sync, &rdev2->flags)) sb->dev_roles[i] = cpu_to_le16(rdev2->raid_disk); + else if (test_bit(WriteCache, &rdev2->flags)) + sb->dev_roles[i] = cpu_to_le16(0xfffd); else if (rdev2->raid_disk >= 0) sb->dev_roles[i] = cpu_to_le16(rdev2->raid_disk); else@@ -5780,7 +5785,8 @@ static int get_disk_info(struct mddev *mddev, void __user * arg)else if (test_bit (In_sync, &rdev->flags)) { info.state |= (1 <<MD_DISK_ACTIVE ); info.state |= (1 <<MD_DISK_SYNC ); - } + }else if (test_bit (WriteCache, &rdev->flags)) + info.state |= (1 <<MD_DISK_WRITECACHE );if (test_bit (WriteMostly, &rdev->flags)) info.state |= (1 <<MD_DISK_WRITEMOSTLY ); }else {@@ -5895,6 +5901,8 @@ static int add_new_disk(struct mddev *mddev, mdu_disk_info_t *info)else clear_bit (WriteMostly, &rdev->flags); +if (info->state & (1 <<MD_DISK_WRITECACHE )) +set_bit (WriteCache, &rdev->flags);/* * check whether the device shows up in other nodes */ @@ -7263,6 +7271,10 @@ static int md_seq_show(struct seq_file *seq, void *v)seq_printf (seq,"(F)" );continue ; } +if (test_bit (WriteCache, &rdev->flags)) { +seq_printf (seq,"(C)" ); +continue ; + }if (rdev->raid_disk <0 )seq_printf (seq,"(S)" );/* spare */ if (test_bit (Replacement, &rdev->flags))drivers/md/md.h +4
-0@@ -175,6 +175,10 @@ enum flag_bits {* This device is seen locally but not * by the whole cluster */ + WriteCache, /* This device is used as write cache. +* Usually, this device should be faster +* than other devices in the array +*/ };#define BB_LEN_MASK (0x00000000000001FFULL)include/uapi/linux/raid/md_p.h +1
-0@@ -89,6 +89,7 @@* read requests will only be sent here in * dire need */ +#define MD_DISK_WRITECACHE 18/* disk is used as the write cache in RAID-5/6 */ typedef struct mdp_device_descriptor_s { __u32 number; /* 0 Device number in the entire set */
PATCH v3 2/8 raid5: directly use mddev->queue
In reply to Shaohua LiWhen the cache layer flushes data from cache disk to raid disks, it will dipsatch IO to raid disks. At that time, we don't have a block device attached to the bio, so directly use mddev->queue. That should not impact IO dispatched to rdev, which has rdev block device attached.
--- drivers/md/raid5.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
drivers/md/raid5.c +5 -7
@@ -223,7 +223,7 @@ static int raid6_idx_to_slot(int idx, struct stripe_head *sh,return slot; } -static void return_io (struct bio *return_bi) +static void return_io (struct r5conf *conf,struct bio *return_bi) {struct bio *bi = return_bi;while (bi) {
@@ -231,8 +231,7 @@ static void return_io(struct bio *return_bi) return_bi = bi->bi_next; bi->bi_next =NULL ; bi->bi_iter.bi_size =0 ; -trace_block_bio_complete (bdev_get_queue (bi->bi_bdev), - bi,0 ); +trace_block_bio_complete (conf->mddev->queue, bi,0 );bio_endio (bi,0 ); bi = return_bi; }
@@ -1200,7 +1199,7 @@ static void ops_complete_biofill(void *stripe_head_ref) }clear_bit (STRIPE_BIOFILL_RUN , &sh->state); -return_io (return_bi); +return_io (sh->raid_conf, return_bi);set_bit (STRIPE_HANDLE , &sh->state);release_stripe (sh);
@@ -4594,7 +4593,7 @@ static void handle_stripe(struct stripe_head *sh)md_wakeup_thread (conf->mddev->thread); } -return_io (s.return_bi); +return_io (conf, s.return_bi);clear_bit_unlock (STRIPE_ACTIVE , &sh->state); }
@@ -5298,8 +5297,7 @@ static void make_request(struct mddev *mddev, struct bio * bi)if ( rw ==WRITE )md_write_end (mddev); -trace_block_bio_complete (bdev_get_queue (bi->bi_bdev), - bi,0 ); +trace_block_bio_complete (mddev->queue, bi,0 );bio_endio (bi,0 ); } }
Signature · 1 line
1.8.1
PATCH v3 4/8 raid5: add some sysfs entries
In reply to Shaohua LiAdd some sysfs entries.
-cache_memory. Control the cache memory size. -cache_reclaim_batch. Control how many stripes reclaim should run in one time.
-cache_memory_watermark. The background reclaim runs if cache memory hits the watermark and stops after hit 1.5x of the watermark. -cache_disk_watermark. The background reclaim runs if cache disk space hits the watermark and stops after hit 1.5x of the watermark. -cache_stat. statistics about cache.
--- drivers/md/raid5-cache.c | 294 ++++++++++++++++++++++++++++++++++++++++++++++- drivers/md/raid5.c | 3 + drivers/md/raid5.h | 1 + 3 files changed, 297 insertions(+), 1 deletion(-)
drivers/md/raid5-cache.c +293 -1
@@ -278,6 +278,12 @@ struct r5c_cache {struct kmem_cache *io_range_kc;struct kmem_cache *stripe_kc;struct bio_set *bio_set; + +atomic64_t in_cache_rq; +atomic64_t out_cache_rq; +atomic64_t in_cache_sectors; +atomic64_t out_cache_sectors; +atomic64_t read_cache_sectors; };enum {
@@ -314,6 +320,12 @@ static inline int r5l_page_blocks(struct r5l_log *log, int pages)return pages << log->page_block_shift; } +static inline int r5l_max_flush_stripes (struct r5l_log *log) +{ +return (log->block_size -sizeof (struct r5l_flush_block )) / +sizeof (__le64); +} +static u32 r5l_calculate_checksum (struct r5l_log *log,u32 crc,void *buf,size_t size,bool data) {
@@ -1644,6 +1656,9 @@ static void r5c_write_bio(struct r5c_cache *cache, struct bio *bio) stripe->existing_pages += new_pages;r5c_unlock_stripe (cache, stripe, &flags); +atomic64_inc (&cache->in_cache_rq); +atomic64_add (bio_sectors (bio), &cache->in_cache_sectors); +if (r5l_queue_bio (&cache->log, bio, r5c_bio_task_end, io_range, reserved_blocks)) goto put_error;
@@ -1692,6 +1707,8 @@ static void r5c_read_bio(struct r5c_cache *cache, struct bio *bio) split = bio;r5c_copy_bio (split, &stripe->data_pages[start], true); +atomic64_add (bio_sectors (split), + &cache->read_cache_sectors);bio_endio (split,0 );
@@ -1837,6 +1854,10 @@ static void r5c_flush_one(struct r5c_cache *cache, struct r5c_stripe *stripe, bio->bi_end_io = r5c_flush_endio; bio->bi_rw =WRITE ;atomic_inc (&stripe->pending_bios); + +atomic64_inc (&cache->out_cache_rq); +atomic64_add (bio_sectors (bio), &cache->out_cache_sectors); +raid5_make_request (cache->mddev, bio); } }
@@ -3102,6 +3123,273 @@ static int r5c_shrink_cache_memory(struct r5c_cache *cache, unsigned long size)return 0 ; } +static ssize_t r5c_show_cache_memory (struct mddev *mddev,char *page) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; + +return sprintf (page,"%lld\n" , cache->max_pages <<PAGE_SHIFT ); +} + +static ssize_t r5c_store_cache_memory (struct mddev *mddev,const char *page, +size_t len) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; +unsigned long new; +LIST_HEAD (page_list); +u64 i; + +if (len >=PAGE_SIZE ) +return -EINVAL ; +if (kstrtoul (page,0 , &new)) +return -EINVAL ; + new >>=PAGE_SHIFT ; + +if (new > cache->max_pages) { + i = cache->max_pages; +while (i < new) { +struct page *page =alloc_page (GFP_KERNEL ); + +if (!page) +break ; +list_add (&page->lru, &page_list); + i++; + } + +spin_lock_irq (&cache->pool_lock); +list_splice (&page_list, &cache->page_pool); + cache->free_pages += i - cache->max_pages; + cache->max_pages = i; + cache->total_pages = i; +r5c_calculate_watermark (cache); +spin_unlock_irq (&cache->pool_lock); +return len; + } +r5c_shrink_cache_memory (cache, new); +return len; +} + +static struct md_sysfs_entry r5c_cache_memory =__ATTR (cache_memory, +S_IRUGO |S_IWUSR , r5c_show_cache_memory, r5c_store_cache_memory); + +int r5c_min_stripe_cache_size (struct r5c_cache *cache) +{ +struct r5conf *conf = cache->mddev->private; +return (conf->chunk_sectors >>PAGE_SECTOR_SHIFT ) * + cache->reclaim_batch; +} + +static void r5c_set_reclaim_batch (struct r5c_cache *cache,int batch) +{ +struct mddev *mddev = cache->mddev; +struct r5conf *conf = mddev->private; +int size; + + size = (cache->stripe_parity_pages <<PAGE_SECTOR_SHIFT ) * batch; +if (size > cache->reserved_space) { + cache->reserved_space = size; +mutex_lock (&cache->log.io_mutex); + cache->log.reserved_blocks =r5l_sector_to_block (&cache->log, + cache->reserved_space) +1 ; +mutex_unlock (&cache->log.io_mutex); +r5c_wake_wait_reclaimer (cache, +RECLAIM_DISK_BACKGROUND ); + }else { +mutex_lock (&cache->log.io_mutex); + cache->log.reserved_blocks -=r5l_sector_to_block (&cache->log, + cache->reserved_space - size); +mutex_unlock (&cache->log.io_mutex); + cache->reserved_space = size; + } + + size = (conf->chunk_sectors >>PAGE_SECTOR_SHIFT ) * batch; + +mddev_lock (mddev); +if (size > conf->max_nr_stripes) +raid5_set_cache_size (mddev, size); +mddev_unlock (mddev); + + cache->reclaim_batch = batch; +} + +static ssize_t r5c_show_cache_reclaim_batch (struct mddev *mddev,char *page) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; + +return sprintf (page,"%d\n" , cache->reclaim_batch); +} + +static ssize_t r5c_store_cache_reclaim_batch (struct mddev *mddev, +const char *page,size_t len) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; +unsigned long new; + +if (len >=PAGE_SIZE ) +return -EINVAL ; +if (kstrtoul (page,0 , &new)) +return -EINVAL ; + +if (new >r5l_max_flush_stripes (&cache->log)) + new =r5l_max_flush_stripes (&cache->log); + +if (new != cache->reclaim_batch) +r5c_set_reclaim_batch (cache, new); +return len; +} + +static struct md_sysfs_entry r5c_cache_reclaim_batch = +__ATTR (cache_reclaim_batch,S_IRUGO |S_IWUSR , + r5c_show_cache_reclaim_batch, r5c_store_cache_reclaim_batch); + +static ssize_t r5c_show_cache_stat (struct mddev *mddev,char *page) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; + +return sprintf (page,"%lld %lld %lld %lld %lld\n" , + (u64 )atomic64_read (&cache->in_cache_rq), + (u64 )atomic64_read (&cache->in_cache_sectors), + (u64 )atomic64_read (&cache->out_cache_rq), + (u64 )atomic64_read (&cache->out_cache_sectors), + (u64 )atomic64_read (&cache->read_cache_sectors)); +} + +static struct md_sysfs_entry r5c_cache_stat = +__ATTR (cache_stat,S_IRUGO , r5c_show_cache_stat,NULL ); + +static ssize_t r5c_show_cache_disk_watermark (struct mddev *mddev,char *page) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; + +return sprintf (page,"%lld\n" , cache->log.low_watermark * + cache->log.block_size); +} + +static ssize_t r5c_store_cache_disk_watermark (struct mddev *mddev, +const char *page,size_t len) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; +struct r5l_log *log = &cache->log; +unsigned long new; + +if (len >=PAGE_SIZE ) +return -EINVAL ; +if (kstrtoul (page,0 , &new)) +return -EINVAL ; + new /= log->block_size; + +if (new *3 /2 >= log->total_blocks) +return -EINVAL ; + +mutex_lock (&log->io_mutex); + log->low_watermark = new; + log->high_watermark = new *3 /2 ; +mutex_unlock (&log->io_mutex); +return len; +} + +static struct md_sysfs_entry r5c_cache_disk_watermark = +__ATTR (cache_disk_watermark,S_IRUGO |S_IWUSR , + r5c_show_cache_disk_watermark, r5c_store_cache_disk_watermark); + +static ssize_t r5c_show_cache_memory_watermark (struct mddev *mddev,char *page) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; + +return sprintf (page,"%lld\n" , cache->low_watermark <<PAGE_SHIFT ); +} + +static ssize_t r5c_store_cache_memory_watermark (struct mddev *mddev, +const char *page,size_t len) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; +unsigned long new; + +if (len >=PAGE_SIZE ) +return -EINVAL ; +if (kstrtoul (page,0 , &new)) +return -EINVAL ; + new >>=PAGE_SHIFT ; + +if (new *2 >= cache->max_pages) +return -EINVAL ; + +spin_lock_irq (&cache->pool_lock); + cache->low_watermark = new; + cache->high_watermark = new <<1 ; +spin_unlock_irq (&cache->pool_lock); +return len; +} + +static struct md_sysfs_entry r5c_cache_memory_watermark = +__ATTR (cache_memory_watermark,S_IRUGO |S_IWUSR , + r5c_show_cache_memory_watermark, r5c_store_cache_memory_watermark); + +static int r5c_init_sysfs (struct r5c_cache *cache) +{ +struct mddev *mddev = cache->mddev; +int ret; + + ret =sysfs_add_file_to_group (&mddev->kobj, &r5c_cache_memory.attr, +NULL ); +if (ret) +return ret; + ret =sysfs_add_file_to_group (&mddev->kobj, + &r5c_cache_reclaim_batch.attr,NULL ); +if (ret) + goto err_reclaim; + ret =sysfs_add_file_to_group (&mddev->kobj, + &r5c_cache_disk_watermark.attr,NULL ); +if (ret) + goto disk_watermark; + ret =sysfs_add_file_to_group (&mddev->kobj, + &r5c_cache_stat.attr,NULL ); +if (ret) + goto stat; + + ret =sysfs_add_file_to_group (&mddev->kobj, + &r5c_cache_memory_watermark.attr,NULL ); +if (ret) + goto memory_watermark; +return 0 ; +memory_watermark: +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_stat.attr,NULL ); +stat: +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_disk_watermark.attr,NULL ); +disk_watermark: +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_reclaim_batch.attr,NULL ); +err_reclaim: +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_memory.attr,NULL ); +return ret; +} + +static void r5c_exit_sysfs (struct r5c_cache *cache) +{ +struct mddev *mddev = cache->mddev; +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_reclaim_batch.attr,NULL ); +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_memory.attr,NULL ); +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_disk_watermark.attr,NULL ); +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_stat.attr,NULL ); +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_memory_watermark.attr,NULL ); +} +static void r5c_free_cache_data (struct r5c_cache *cache) {struct r5c_stripe *stripe;
@@ -3212,8 +3500,11 @@ struct r5c_cache *r5c_init_cache(struct r5conf *conf, struct md_rdev *rdev) cache->reclaim_thread->timeout =CHECKPOINT_TIMEOUT ;r5c_shrink_cache_memory (cache, cache->max_pages); - +if (r5c_init_sysfs (cache)) + goto err_sysfs;return cache; +err_sysfs: +md_unregister_thread (&cache->reclaim_thread); err_page:r5c_free_cache_data (cache);
@@ -3232,6 +3523,7 @@ struct r5c_cache *r5c_init_cache(struct r5conf *conf, struct md_rdev *rdev)void r5c_exit_cache (struct r5c_cache *cache) { +r5c_exit_sysfs (cache);md_unregister_thread (&cache->reclaim_thread);r5l_exit_log (&cache->log);
drivers/md/raid5.c +3 -0
@@ -5907,6 +5907,9 @@ raid5_set_cache_size(struct mddev *mddev, int size)if (size <=16 || size >32768 )return -EINVAL ; +if (conf->cache && size <r5c_min_stripe_cache_size (conf->cache)) + size =r5c_min_stripe_cache_size (conf->cache); + conf->min_nr_stripes = size;while (size < conf->max_nr_stripes &&drop_one_stripe (conf))
drivers/md/raid5.h +1 -0
@@ -617,4 +617,5 @@ void r5c_exit_cache(struct r5c_cache *cache);void r5c_write_start (struct mddev *mddev,struct bio *bi);void r5c_write_end (struct mddev *mddev,struct bio *bi);void r5c_quiesce (struct r5conf *conf,int state); +int r5c_min_stripe_cache_size (struct r5c_cache *cache);#endif
Signature · 1 line
1.8.1
PATCH v3 5/8 md: don't allow resize/reshape with cache support
In reply to Shaohua LiIf cache support is enabled, don't allow resize/reshape in current stage. In the future, we can flush all data from cache to raid before resize/reshape and then allow resize/reshape.
--- drivers/md/raid5.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/md/raid5.c +6 -0
@@ -7207,6 +7207,10 @@ static int raid5_resize(struct mddev *mddev, sector_t sectors)* worth it. */ sector_t newsize; +struct r5conf *conf = mddev->private; + +if (conf->cache) +return -EINVAL ; sectors &= ~((sector_t )mddev->chunk_sectors -1 ); newsize =raid5_size (mddev, sectors, mddev->raid_disks); if (mddev->external_size &&
@@ -7258,6 +7262,8 @@ static int check_reshape(struct mddev *mddev) {struct r5conf *conf = mddev->private; +if (conf->cache) +return -EINVAL ;if (mddev->delta_disks ==0 && mddev->new_layout == mddev->layout && mddev->new_chunk_sectors == mddev->chunk_sectors)
Signature · 1 line
1.8.1
Re: [PATCH v3 5/8] md: don't allow resize/reshape with cache support
Quoted 25 lines · On Wed, 3 Jun 2015 15:48:40 -0700 Shaohua Li <[email protected]> wrote:
If cache support is enabled, don't allow resize/reshape in current stage. In the future, we can flush all data from cache to raid before resize/reshape and then allow resize/reshape.
Signed-off-by Shaohua Li <[email protected]>--- drivers/md/raid5.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/md/raid5.c +4
-0@@ -7207,6 +7207,10 @@ static int raid5_resize(struct mddev *mddev, sector_t sectors)* worth it. */ sector_t newsize; +struct r5conf *conf = mddev->private; + +if (conf->cache) +return -EINVAL ; sectors &= ~((sector_t )mddev->chunk_sectors -1 ); newsize =raid5_size (mddev, sectors, mddev->raid_disks); if (mddev->external_size &&
drivers/md/raid5.c static int check_reshape(struct mddev *mddev) · line 7269
Quoted 8 lines
drivers/md/raid5.c +2
-0@@ -7258,6 +7262,8 @@ static int check_reshape(struct mddev *mddev) {struct r5conf *conf = mddev->private; +if (conf->cache) +return -EINVAL ;if (mddev->delta_disks ==0 && mddev->new_layout == mddev->layout && mddev->new_chunk_sectors == mddev->chunk_sectors)
This patch should come before that patch which enables caches - the could should be correct at each point in the series.
NeilBrown
PATCH v3 6/8 raid5: skip resync if caching is enabled
In reply to Shaohua LiIf caching is enabled, the caching layer will guarantee data consistency, so skip resync for unclean shutdown
--- drivers/md/raid5.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/md/raid5.c +7 -0
@@ -6975,6 +6975,13 @@ static int run(struct mddev *mddev)if (mddev->queue)blk_queue_logical_block_size (mddev->queue,STRIPE_SIZE ); conf->skip_copy =1 ; + +if (mddev->recovery_cp ==0 ) { +printk (KERN_NOTICE +"md/raid:%s: skip resync with caching enabled\n" , +mdname (mddev)); + mddev->recovery_cp = MaxSector; + } }return 0 ;
Signature · 1 line
1.8.1
PATCH v3 7/8 raid5: guarantee cache release stripes in correct way
In reply to Shaohua LiWhen cache trys to flush data to raid, we must be very careful about stripe release. The issue is if we release a stripe and the stripe is handling by raid5d and we add a new bio into the stripe later, the stripe will enter the raid5 state machine several times, which r5cache_write_parity can't handle well. This could happen if get_active_stripe() sleeps in adding bio to several stripes. To solve this issue, we guarantee stripe release after all bio are added to corresponding stripes. This is a performance win too if the bug happens.
--- drivers/md/raid5-cache.c | 28 +++++++++------- drivers/md/raid5.c | 84 +++++++++++++++++++++++++++++++----------------- drivers/md/raid5.h | 11 ++++++- 3 files changed, 80 insertions(+), 43 deletions(-)
drivers/md/raid5-cache.c +16 -12
@@ -1670,7 +1670,7 @@ static void r5c_write_bio(struct r5c_cache *cache, struct bio *bio)r5c_enter_error_mode (cache, -ENOMEM );r5c_check_wait_error_mode (cache); error_mode: -raid5_make_request (cache->mddev, bio); +raid5_make_request (cache->mddev, bio,NULL ); }static void r5c_read_bio (struct r5c_cache *cache,struct bio *bio)
@@ -1686,7 +1686,7 @@ static void r5c_read_bio(struct r5c_cache *cache, struct bio *bio) stripe =r5c_get_stripe (cache, stripe_index);if (!stripe) { -raid5_make_request (cache->mddev, bio); +raid5_make_request (cache->mddev, bio,NULL );return ; }
@@ -1725,7 +1725,7 @@ static void r5c_read_bio(struct r5c_cache *cache, struct bio *bio) }else split = bio; -raid5_make_request (cache->mddev, split); +raid5_make_request (cache->mddev, split,NULL ); start = tmp; }
@@ -1823,7 +1823,7 @@ void r5c_write_end(struct mddev *mddev, struct bio *bi) }static void r5c_flush_one (struct r5c_cache *cache,struct r5c_stripe *stripe, -int start,int pages) +int start,int pages,struct raid5_plug_context *pc) {sector_t base;struct bio *bio;
@@ -1858,7 +1858,7 @@ static void r5c_flush_one(struct r5c_cache *cache, struct r5c_stripe *stripe,atomic64_inc (&cache->out_cache_rq);atomic64_add (bio_sectors (bio), &cache->out_cache_sectors); -raid5_make_request (cache->mddev, bio); +raid5_make_request (cache->mddev, bio, pc); } }
@@ -1871,7 +1871,8 @@ static void r5c_put_stripe_dirty(struct r5c_cache *cache, } } -static void r5c_flush_stripe (struct r5c_cache *cache,struct r5c_stripe *stripe) +static void r5c_flush_stripe (struct r5c_cache *cache,struct r5c_stripe *stripe, +struct raid5_plug_context *pc) {unsigned long *stripe_bits;int chunk_stripes;
@@ -1900,7 +1901,7 @@ static void r5c_flush_stripe(struct r5c_cache *cache, struct r5c_stripe *stripe)while (end < cache->stripe_data_pages && stripe->data_pages[end]) end++; -r5c_flush_one (cache, stripe, start, end - start); +r5c_flush_one (cache, stripe, start, end - start, pc); }r5c_put_stripe_dirty (cache, stripe); }
@@ -2034,11 +2035,14 @@ static void r5c_reclaim_stripe_list(struct r5c_cache *cache,struct r5c_io_range *range;u64 seq;sector_t meta; -struct blk_plug plug;size_t size =0 ; +struct raid5_plug_context pc;if (list_empty (stripe_list))return ; + +raid5_context_init (&pc); +list_sort (NULL , stripe_list, r5c_stripe_list_cmp);list_for_each_entry (stripe, stripe_list, lru) { cache->stripe_flush_data[size] =
@@ -2047,11 +2051,11 @@ static void r5c_reclaim_stripe_list(struct r5c_cache *cache, } size *=sizeof (__le64); -blk_start_plug (&plug);/* step 1: start write to raid */ list_for_each_entry (stripe, stripe_list, lru) -r5c_flush_stripe (cache, stripe); -blk_finish_plug (&plug); +r5c_flush_stripe (cache, stripe, &pc); + +raid5_context_unplug (&pc, cache->mddev, false);/* step 2: wait parity write to cache */ list_for_each_entry_reverse (stripe, stripe_list, lru)
@@ -2158,7 +2162,7 @@ static void r5c_reclaim_thread(struct md_thread *thread)wake_up (&cache->error_wait);while ((bio =bio_list_pop (&cache->retry_bio_list)) !=NULL ) -raid5_make_request (cache->mddev, bio); +raid5_make_request (cache->mddev, bio,NULL );if (++cache->retry_cnt <MAX_RETRY ) { cache->next_retry_time = jiffies +
drivers/md/raid5.c +54 -30
150 lines not on this page · raw message
drivers/md/raid5.h +10 -1
@@ -607,8 +607,17 @@ static inline int algorithm_is_DDF(int layout)extern void md_raid5_kick_device (struct r5conf *conf);extern int raid5_set_cache_size (struct mddev *mddev,int size); +struct raid5_plug_context { +struct list_head list; +struct list_head temp_inactive_list[NR_STRIPE_HASH_LOCKS ]; +}; +void raid5_context_init (struct raid5_plug_context *context); +void raid5_context_unplug (struct raid5_plug_context *context, +struct mddev *mddev,bool from_schedule); +void release_stripe (struct stripe_head *sh); -void raid5_make_request (struct mddev *mddev,struct bio *bi); +void raid5_make_request (struct mddev *mddev,struct bio *bi, +struct raid5_plug_context *context);void r5c_handle_bio (struct r5c_cache *cache,struct bio *bi);int r5c_write_parity (struct r5c_cache *cache,struct stripe_head *sh);void r5c_flush_pending_parity (struct r5c_cache *cache);
Signature · 1 line
1.8.1
PATCH v3 8/8 raid5: multi-thread support for raid5 caching reclaim
In reply to Shaohua LiThere are several stalled stages in raid5 caching reclaim, which can significantly harm reclaim performance. To mitigate the performance issue, we introduce multi-thread support for reclaim. Since each thread records reclaimed stripes in flush start/end block, it's safe for multi-thread.
--- drivers/md/raid5-cache.c | 343 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 288 insertions(+), 55 deletions(-)
drivers/md/raid5-cache.c +288 -55
@@ -224,6 +224,8 @@ enum { };#define STRIPE_LOCK_BITS 8 + +#define MAX_RECLAIM_WORKERS 16struct r5c_cache {struct mddev *mddev;struct md_rdev *rdev;
@@ -258,9 +260,11 @@ struct r5c_cache {unsigned long reclaim_reason;wait_queue_head_t reclaim_wait;struct md_thread *reclaim_thread; -__le64 *stripe_flush_data;int quiesce_state; +struct md_thread *reclaim_workers[MAX_RECLAIM_WORKERS ]; +int reclaim_worker_cnt; +int in_recovery;struct work_struct pending_io_work;
@@ -294,9 +298,8 @@ enum {RECLAIM_DISK_BACKGROUND =9 ,/* try to reclaim disk */ RECLAIM_FLUSH_ALL =16 ,/* flush all data to raid */ -QUIESCE_NONE =0 , +QUIESCE_END =0 ,QUIESCE_START =1 , -QUIESCE_END =2 ,ERROR_NOERROR =0 , ERROR_PREPARE = 1, /* Had an error, flushing cache to raid */
@@ -1958,7 +1961,9 @@ static void r5c_select_stripes(struct r5c_cache *cache, struct list_head *list) {int stripes;bool blocking; +static DEFINE_MUTEX (lock); +mutex_lock (&lock);/* * generally select full stripe, if no disk space, select first stripe */
@@ -1991,6 +1996,7 @@ static void r5c_select_stripes(struct r5c_cache *cache, struct list_head *list) }spin_unlock_irq (&cache->tree_lock); +mutex_unlock (&lock); }static void r5c_disks_flush_end (struct bio *bio,int err)
@@ -2028,11 +2034,44 @@ static int r5c_stripe_list_cmp(void *priv, struct list_head *a,return !(stripe_a->raid_index < stripe_b->raid_index); } -static void r5c_reclaim_stripe_list (struct r5c_cache *cache, -struct list_head *stripe_list) +struct r5c_reclaim_context { +struct list_head stripe_list; +__le64 *stripe_flush_data; +u64 seq; +sector_t meta; +struct completion comp; +}; + +static struct r5c_reclaim_context * +r5c_alloc_reclaim_context (struct r5c_cache *cache) +{ +struct r5c_reclaim_context *context; + + context =kzalloc (sizeof (*context),GFP_KERNEL ); +if (!context) +return NULL ; +INIT_LIST_HEAD (&context->stripe_list); + context->stripe_flush_data =kmalloc (PAGE_SIZE ,GFP_KERNEL ); +if (!context->stripe_flush_data) { +kfree (context); +return NULL ; + } +init_completion (&context->comp); +return context; +} + +static void r5c_free_reclaim_context (struct r5c_reclaim_context *context) +{ +kfree (context->stripe_flush_data); +kfree (context); +} + +static void r5c_do_reclaim (struct r5c_cache *cache, +struct r5c_reclaim_context *context) { +struct list_head *stripe_list = &context->stripe_list; +__le64 *stripe_flush_data = context->stripe_flush_data;struct r5c_stripe *stripe; -struct r5c_io_range *range;u64 seq;sector_t meta;size_t size =0 ;
@@ -2045,7 +2084,7 @@ static void r5c_reclaim_stripe_list(struct r5c_cache *cache,list_sort (NULL , stripe_list, r5c_stripe_list_cmp);list_for_each_entry (stripe, stripe_list, lru) { - cache->stripe_flush_data[size] = + stripe_flush_data[size] =cpu_to_le64 (stripe->raid_index); size++; }
@@ -2063,7 +2102,7 @@ static void r5c_reclaim_stripe_list(struct r5c_cache *cache,/* step 3: make sure data and parity settle down */ r5l_flush_block (&cache->log,R5LOG_TYPE_FLUSH_START , - cache->stripe_flush_data, size, &seq, &meta); + stripe_flush_data, size, &seq, &meta);/* step 4: continue write to raid */ list_for_each_entry (stripe, stripe_list, lru) {
@@ -2087,7 +2126,7 @@ static void r5c_reclaim_stripe_list(struct r5c_cache *cache,/* step 7: mark data is flushed to raid */ r5l_flush_block (&cache->log,R5LOG_TYPE_FLUSH_END , - cache->stripe_flush_data, size, &seq, &meta); + stripe_flush_data, size, &seq, &meta);/* step 8: mark stripe as dead */ while (!list_empty (stripe_list)) {
@@ -2100,6 +2139,23 @@ static void r5c_reclaim_stripe_list(struct r5c_cache *cache,r5c_put_stripe (stripe); } + context->seq = seq; + context->meta = meta; +} + +static void r5c_do_reclaim_and_write_super (struct r5c_cache *cache, +struct r5c_reclaim_context *context) +{ +struct r5c_io_range *range; +u64 seq; +sector_t meta; + +if (list_empty (&context->stripe_list)) +return ; + +r5c_do_reclaim (cache, context); + seq = context->seq; + meta = context->meta;/* step 9: advance superblock checkpoint */ spin_lock_irq (&cache->tree_lock);
@@ -2118,14 +2174,67 @@ static void r5c_reclaim_stripe_list(struct r5c_cache *cache,r5l_write_super (&cache->log, seq, meta); } +static void r5c_reclaim_work (struct r5c_cache *cache, +struct r5c_reclaim_context *context) +{ +/* +* select stripe will freeze stripe, which will guarantee no +* new task pending in error mode +* */ +r5c_select_stripes (cache, &context->stripe_list); + +r5c_do_reclaim (cache, context); +} + +static void r5c_reclaim_stripe_list (struct r5c_cache *cache, +struct list_head *stripe_list) +{ +int size; +struct r5c_reclaim_context *context; + + context =r5c_alloc_reclaim_context (cache); + +while (!list_empty (stripe_list)) { + size =0 ; +while (!list_empty (stripe_list) && size <RECLAIM_BATCH ) { +list_move_tail (stripe_list->next, + &context->stripe_list); + size++; + } +if (!list_empty (stripe_list)) +r5c_do_reclaim (cache, context); +else +r5c_do_reclaim_and_write_super (cache, context); + } + +r5c_free_reclaim_context (context); +} + +static void r5c_reclaim_thread_check_quiesce (struct r5c_cache *cache, +struct r5c_reclaim_context *context) +{ +if (cache->quiesce_state ==QUIESCE_START ) { +complete (&context->comp); +wait_event (cache->reclaim_wait, cache->quiesce_state == +QUIESCE_END ); +reinit_completion (&context->comp); + } +} +static void r5c_reclaim_thread (struct md_thread *thread) {struct mddev *mddev = thread->mddev;struct r5conf *conf = mddev->private;struct r5c_cache *cache = conf->cache; -LIST_HEAD (stripe_list); +struct r5c_reclaim_context *context = thread->private;bool retry; +r5c_reclaim_thread_check_quiesce (cache, context); + +if (thread != cache->reclaim_thread) { +r5c_reclaim_work (cache, context); +return ; + } do_retry: retry = false;
@@ -2134,9 +2243,9 @@ static void r5c_reclaim_thread(struct md_thread *thread)* select stripe will freeze stripe, which will guarantee no * new task pending in error mode * */ -r5c_select_stripes (cache, &stripe_list); +r5c_select_stripes (cache, &context->stripe_list); -if (list_empty (&stripe_list)) { +if (list_empty (&context->stripe_list)) {clear_bit (RECLAIM_MEM_FULL , &cache->reclaim_reason);clear_bit (RECLAIM_MEM_BACKGROUND , &cache->reclaim_reason);
@@ -2144,7 +2253,7 @@ static void r5c_reclaim_thread(struct md_thread *thread) &cache->reclaim_reason);clear_bit (RECLAIM_FLUSH_ALL , &cache->reclaim_reason); }else -r5c_reclaim_stripe_list (cache, &stripe_list); +r5c_do_reclaim_and_write_super (cache, context);wake_up (&cache->reclaim_wait); }
@@ -2179,20 +2288,80 @@ static void r5c_reclaim_thread(struct md_thread *thread) cache->error_state =0 ;mddev_resume (cache->mddev); } -if (cache->quiesce_state ==QUIESCE_START ) { -/* user IO already finished, we just stop reclaim */ - cache->reclaim_reason =0 ; - cache->quiesce_state =QUIESCE_END ; -wake_up (&cache->reclaim_wait); -wait_event (cache->reclaim_wait, cache->quiesce_state == -QUIESCE_NONE ); +} + +static struct md_thread *r5c_init_reclaim_thread (struct r5c_cache *cache) +{ +struct r5c_reclaim_context *context; +struct md_thread *thread; + + context =r5c_alloc_reclaim_context (cache); +if (!context) +return NULL ; + + thread =md_register_thread (r5c_reclaim_thread, + cache->mddev,"reclaim" ); +if (!thread) { +r5c_free_reclaim_context (context); +return NULL ; + } + thread->private = context; + +return thread; +} + +static void r5c_exit_reclaim_thread (struct r5c_cache *cache, +struct md_thread **thread) +{ +struct r5c_reclaim_context *context; + + context = (*thread)->private; +r5c_free_reclaim_context (context); + +md_unregister_thread (thread); +} + +static int r5c_init_reclaimers (struct r5c_cache *cache) +{ +struct md_thread *thread; + + thread =r5c_init_reclaim_thread (cache); +if (!thread) +return -ENOMEM ; + cache->reclaim_thread = thread; + cache->reclaim_thread->timeout =CHECKPOINT_TIMEOUT ; +return 0 ; +} + +static void r5c_exit_reclaimers (struct r5c_cache *cache) +{ +int i = cache->reclaim_worker_cnt; + +while (i >0 ) { +r5c_exit_reclaim_thread (cache, &cache->reclaim_workers[i -1 ]); + i--; } + +r5c_exit_reclaim_thread (cache, &cache->reclaim_thread); +} + +static void r5c_wakeup_reclaimer_threads (struct r5c_cache *cache) +{ +int i; + +md_wakeup_thread (cache->reclaim_thread); + +preempt_disable (); +for (i =0 ; i < cache->reclaim_worker_cnt; i++) +if (cache->reclaim_workers[i]) +md_wakeup_thread (cache->reclaim_workers[i]); +preempt_enable (); }static void r5c_wake_reclaimer (struct r5c_cache *cache,int reason) {set_bit (reason, &cache->reclaim_reason); -md_wakeup_thread (cache->reclaim_thread); +r5c_wakeup_reclaimer_threads (cache); }static void r5c_wake_wait_reclaimer (struct r5c_cache *cache,int reason)
@@ -2205,17 +2374,25 @@ static void r5c_wake_wait_reclaimer(struct r5c_cache *cache, int reason)void r5c_quiesce (struct r5conf *conf,int state) {struct r5c_cache *cache = conf->cache; +struct r5c_reclaim_context *context; +int i;if (!cache || cache->error_state)return ;if (state ==1 ) {r5c_wake_wait_reclaimer (cache,RECLAIM_FLUSH_ALL ); + cache->quiesce_state =QUIESCE_START ; -md_wakeup_thread (cache->reclaim_thread); -wait_event (cache->reclaim_wait, cache->quiesce_state == -QUIESCE_END ); +r5c_wakeup_reclaimer_threads (cache); + +for (i =0 ; i < cache->reclaim_worker_cnt; i++) { + context = cache->reclaim_workers[i]->private; +wait_for_completion (&context->comp); + } + context = cache->reclaim_thread->private; +wait_for_completion (&context->comp); }else if (state ==0 ) { - cache->quiesce_state =QUIESCE_NONE ; + cache->quiesce_state =QUIESCE_END ;wake_up (&cache->reclaim_wait); } }
@@ -2751,7 +2928,6 @@ static int r5c_recover_stripes(struct r5c_load_ctx *ctx) {struct r5c_cache *cache = ctx->cache;LIST_HEAD (list); -int i;r5l_check_stripes_checksum (ctx);
@@ -2762,18 +2938,7 @@ static int r5c_recover_stripes(struct r5c_load_ctx *ctx) cache->in_recovery =1 ; -while (!list_empty (&ctx->stripes_with_parity)) { - i =0 ; -/* Can't handle large stripe list */ -while (i <RECLAIM_BATCH && - !list_empty (&ctx->stripes_with_parity)) { -list_move_tail (ctx->stripes_with_parity.next, - &list); - i++; - } -r5c_reclaim_stripe_list (cache, &list); -BUG_ON (!list_empty (&list)); - } +r5c_reclaim_stripe_list (cache, &ctx->stripes_with_parity); cache->in_recovery =0 ;return 0 ;
@@ -3181,16 +3346,20 @@ int r5c_min_stripe_cache_size(struct r5c_cache *cache) {struct r5conf *conf = cache->mddev->private;return (conf->chunk_sectors >>PAGE_SECTOR_SHIFT ) * - cache->reclaim_batch; + cache->reclaim_batch * (1 + cache->reclaim_worker_cnt); } -static void r5c_set_reclaim_batch (struct r5c_cache *cache,int batch) +static void r5c_set_reclaim_batch (struct r5c_cache *cache,int batch, +int threads) {struct mddev *mddev = cache->mddev;struct r5conf *conf = mddev->private;int size; - size = (cache->stripe_parity_pages <<PAGE_SECTOR_SHIFT ) * batch; + threads++; + + size = (cache->stripe_parity_pages <<PAGE_SECTOR_SHIFT ) * batch * + threads;if (size > cache->reserved_space) { cache->reserved_space = size;mutex_lock (&cache->log.io_mutex);
@@ -3207,7 +3376,7 @@ static void r5c_set_reclaim_batch(struct r5c_cache *cache, int batch) cache->reserved_space = size; } - size = (conf->chunk_sectors >>PAGE_SECTOR_SHIFT ) * batch; + size = (conf->chunk_sectors >>PAGE_SECTOR_SHIFT ) * batch * threads;mddev_lock (mddev);if (size > conf->max_nr_stripes)
@@ -3241,7 +3410,7 @@ static ssize_t r5c_store_cache_reclaim_batch(struct mddev *mddev, new =r5l_max_flush_stripes (&cache->log);if (new != cache->reclaim_batch) -r5c_set_reclaim_batch (cache, new); +r5c_set_reclaim_batch (cache, new, cache->reclaim_worker_cnt);return len; }
@@ -3337,6 +3506,68 @@ static struct md_sysfs_entry r5c_cache_memory_watermark =__ATTR (cache_memory_watermark,S_IRUGO |S_IWUSR , r5c_show_cache_memory_watermark, r5c_store_cache_memory_watermark); +static ssize_t r5c_show_reclaim_threads (struct mddev *mddev,char *page) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; + +return sprintf (page,"%d\n" , cache->reclaim_worker_cnt); +} + +static void r5c_set_reclaim_thread_count (struct r5c_cache *cache,int cnt) +{ +struct md_thread *thread; +int old_cnt; + +if (cache->reclaim_worker_cnt == cnt) +return ; +if (cnt >MAX_RECLAIM_WORKERS ) + cnt =MAX_RECLAIM_WORKERS ; + + old_cnt = cache->reclaim_worker_cnt; +if (old_cnt > cnt) { + cache->reclaim_worker_cnt = cnt; +/* make sure r5c_wake_reclaimer() isn't using thread */ +synchronize_sched (); + } + +while (old_cnt > cnt) { +r5c_exit_reclaim_thread (cache, + &cache->reclaim_workers[old_cnt -1 ]); + old_cnt--; + } +while (old_cnt < cnt) { + thread =r5c_init_reclaim_thread (cache); +if (!thread) +break ; + cache->reclaim_workers[old_cnt++] = thread; + } + +r5c_set_reclaim_batch (cache, cache->reclaim_batch, old_cnt); + + cache->reclaim_worker_cnt = old_cnt; +} + +static ssize_t r5c_store_reclaim_threads (struct mddev *mddev, +const char *page,size_t len) +{ +struct r5conf *conf = mddev->private; +struct r5c_cache *cache = conf->cache; +unsigned int new; + +if (len >=PAGE_SIZE ) +return -EINVAL ; +if (kstrtouint (page,0 , &new)) +return -EINVAL ; + +r5c_set_reclaim_thread_count (cache, new); +return len; +} + +static struct md_sysfs_entry r5c_cache_reclaim_threads = +__ATTR (cache_reclaim_threads,S_IRUGO |S_IWUSR , + r5c_show_reclaim_threads, r5c_store_reclaim_threads); +static int r5c_init_sysfs (struct r5c_cache *cache) {struct mddev *mddev = cache->mddev;
@@ -3363,7 +3594,16 @@ static int r5c_init_sysfs(struct r5c_cache *cache) &r5c_cache_memory_watermark.attr, NULL);if (ret) goto memory_watermark; + + ret =sysfs_add_file_to_group (&mddev->kobj, + &r5c_cache_reclaim_threads.attr,NULL ); +if (ret) + goto reclaim_threads; +return 0 ; +reclaim_threads: +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_memory_watermark.attr,NULL ); memory_watermark:sysfs_remove_file_from_group (&mddev->kobj, &r5c_cache_stat.attr,NULL );
@@ -3392,6 +3632,8 @@ static void r5c_exit_sysfs(struct r5c_cache *cache) &r5c_cache_stat.attr, NULL);sysfs_remove_file_from_group (&mddev->kobj, &r5c_cache_memory_watermark.attr,NULL ); +sysfs_remove_file_from_group (&mddev->kobj, + &r5c_cache_reclaim_threads.attr,NULL ); }static void r5c_free_cache_data (struct r5c_cache *cache)
@@ -3446,10 +3688,6 @@ struct r5c_cache *r5c_init_cache(struct r5conf *conf, struct md_rdev *rdev) cache->stripe_parity_pages = (cache->stripe_size - cache->stripe_data_size) >>PAGE_SECTOR_SHIFT ; - cache->stripe_flush_data =kmalloc (PAGE_SIZE ,GFP_KERNEL ); -if (!cache->stripe_flush_data) - goto io_range_kc; - cache->io_range_kc =KMEM_CACHE (r5c_io_range,0 );if (!cache->io_range_kc) goto io_range_kc;
@@ -3497,18 +3735,15 @@ struct r5c_cache *r5c_init_cache(struct r5conf *conf, struct md_rdev *rdev)r5c_calculate_watermark (cache); - cache->reclaim_thread =md_register_thread (r5c_reclaim_thread, - mddev,"reclaim" ); -if (!cache->reclaim_thread) +if (r5c_init_reclaimers (cache)) goto err_page; - cache->reclaim_thread->timeout =CHECKPOINT_TIMEOUT ;r5c_shrink_cache_memory (cache, cache->max_pages);if (r5c_init_sysfs (cache)) goto err_sysfs;return cache; err_sysfs: -md_unregister_thread (&cache->reclaim_thread); +r5c_exit_reclaimers (cache); err_page:r5c_free_cache_data (cache);
@@ -3520,7 +3755,6 @@ struct r5c_cache *r5c_init_cache(struct r5conf *conf, struct md_rdev *rdev) stripe_kc:kmem_cache_destroy (cache->io_range_kc); io_range_kc: -kfree (cache->stripe_flush_data);kfree (cache);return NULL ; }
@@ -3528,7 +3762,7 @@ struct r5c_cache *r5c_init_cache(struct r5conf *conf, struct md_rdev *rdev)void r5c_exit_cache (struct r5c_cache *cache) {r5c_exit_sysfs (cache); -md_unregister_thread (&cache->reclaim_thread); +r5c_exit_reclaimers (cache);r5l_exit_log (&cache->log);r5c_free_cache_data (cache);
@@ -3537,6 +3771,5 @@ void r5c_exit_cache(struct r5c_cache *cache)kmem_cache_destroy (cache->stripe_kc);kmem_cache_destroy (cache->io_range_kc); -kfree (cache->stripe_flush_data);kfree (cache); }
Signature · 1 line
1.8.1
v3 cover MD: a caching layer for raid5/6
In reply to Shaohua LiSorry for intruding on the patch reviewing process and for the novice question.
Is this caching layer also suitable for a smaller system with only one raid5 where /boot and / resides? Or is it meant for dedicated data storage(s) only?
Regards
Davor Vusir
-------- Vidarebefordrat meddelande --------
Ämne: [PATCH v3 0/8] MD: a caching layer for raid5/6
Datum: Wed, 3 Jun 2015 15:48:35 -0700
Från: Shaohua Li <[email protected]>
Till: [email protected]
Kopia: [email protected], [email protected], [email protected], [email protected], [email protected]
Hi,
This is the third version of the raid5/6 caching layer patches. The patches add a caching layer for raid5/6. The caching layer uses a SSD as a cache for a raid 5/6. It works like the similar way of a hardware raid controller. The purpose is to improve raid performance (reduce read-modify-write) and fix write hole issue. The main patch is patch 3 and the description has all details about the implementation. Please review!
Thanks,
Shaohua
V3:
-make reclaim multi-thread
-add statistics in sysfs
-bug fixes
V2:
-metadata write doesn't use FUA
-discard request is only issued when necessary
-bug fixes and cleanup
Shaohua Li (7):
raid5: directly use mddev->queue
raid5: A caching layer for RAID5/6
raid5: add some sysfs entries
md: don't allow resize/reshape with cache support
raid5: skip resync if caching is enabled
raid5: guarantee cache release stripes in correct way
raid5: multi-thread support for raid5 caching reclaim
Song Liu (1):
MD: add a new disk role to present cache device
drivers/md/Makefile | 2 +- drivers/md/md.c | 14 +- drivers/md/md.h | 4 + drivers/md/raid5-cache.c | 3775 ++++++++++++++++++++++++++++++++++++++++ drivers/md/raid5.c | 177 +- drivers/md/raid5.h | 25 +- include/uapi/linux/raid/md_p.h | 73 + 7 files changed, 4022 insertions(+), 48 deletions(-) create mode 100644 drivers/md/raid5-cache.c
Signature · 12 lines
1.8.1 -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): c6df8779f11a4dc3362a04e7cee0be2aec213ebe.1433356864.git.shli@fb.com
Re: [PATCH v3 3/8] raid5: A caching layer for RAID5/6
On Wed, 3 Jun 2015 15:48:38 -0700 Shaohua Li <[email protected]> wrote:
Hi,
sorry for the delay in getting to this.
3000 lines of code is rather hard to review - especially with so few comments :-)
There seem to be a number of fairly well defined modules in the code, such as managing the various data structures. Maybe if these arrived one per patch it would be easier to review them individually.
Quoted 20 lines
Main goal of the caching layer is to aggregate IO write to hopefully make full stripe IO write and fix write hole issue. This might speed up read too, but it's not optimized for read, eg, we don't proactively cache data for read. The aggregation makes a lot of sense for workloads which sequentially write to several files with/without fsync. Such workloads are popular in today's datacenter.
Write IO data will write to a cache disk (SSD) first, then later the data will be flushed to raid disks.
The cache disk will be organized as a simple ring buffer log. For IO data, a tuple (
raid_sector, io length, checksum, data) will be appended to the log; for raid 5/6 parity, a tuple (stripe_sector, parity length, checksum, parity data) will be appended to the log. We don't have on-disk index for the data appended to the log. So either we can rebuild an in-memory index at startup with scanning the whole log, or we can flush all data from cache disk to raid disks at shutdown so cache disk has no valid data. Current code chooses the second option, but this can be easily changed.
Quoted 2 lines
We have a simple meta data for above tuples. It's essentially a tuple (sequence, metadata length).
How are these tuples store in the log? One tuples per block? A block with lots of tuples followed by all the described data? Or does the block of tuples follow the data?
Quoted 6 lines
Crash recovery or startup will scan
the log to read the metadata and rebuild in-memory index. If metadata is broken at the head of the log, even metadata afterward is ok, the scanning will not work well. So we take some steps to mitigate the issue:
-A flush request is sent to cache disk every second to relieve the issue
I don't feel at all comfortable about the every-second flush. I can certainly understand a flush after 50% of the log space is written, or even 25%. but time-based flushes should be coming from the filesystem.
Quoted 2 lines
-FLUSH/FUA request is carefully handled. FLUSH/FUA will only be dispatched after all previous requests finish.
This certainly makes sense.
Quoted 6 lines
The in-memory index is a simple list of
io_range(sequence, metadata sector, data sector, length). The list is orded by sequence. The firstio_rangeentry's metadata sector is the tail of the log. There is also a struct to track io ranges within a stripe. All stripes will be organized as a radix tree.
It would be useful here to briefly say how these data structures are used.
I assumed the order-by-sequence list is to flush data to RAID when the log is getting full?
The radix-tree tracks where in the log each stripe is - is that correct? So it is easy to find the stripe at the tail of the list to flush it out.
Quoted 7 lines
All IO data will be copied to a memory pool for caching too until the data is flushed to raid disks. This is just to simplify the implementation, it's not mandated. In the future flush can do a data copy from cache disk to raid disks, so the memory pool can be discarded. If a stripe is flushed to raid disks, memory of the stripe can be reused.
Again, why a separate memory pool rather than just leaving the data in the stripe cache until it is safe in the RAID?
Quoted 7 lines
We have two limited resources here, memory pool and cache disk space. If resource is tight, we will do reclaim. In either case, we will flush some data from cache disk to raid disks. However, we implement different strategies. For memory reclaim, we prefer reclaiming full stripe. For cache disk space reclaim, we prefer reclaiming
io_rangeentry at the head of index list.
Wouldn't full stripes be scheduled to the RAID immediately they become full (or immediately after the parity is safe in the log)? So for memory reclaim it would make sense to prefer stripes that have been idle for a long time - so an LRU list.
Certainly when the log approaches full you need to start flushing things at the start of the log.
Quoted 6 lines
If cache disk has IO error, since all data are in memory pool, we will flush all data to raid disks and fallback to no-cache-disk mode. IO error of cache disk doesn't corrupt any data. After some time, we will try to use cache disk again if the disk is ok. The retry will stop after several rounds of failure.
The normal data flow involves lots of writing to the cache device and very little if any reading. So we will probably need some sort of "scrub" process to read and verify the log occasionally, just to be sure that reads still work. That could be largely done in user-space.
Quoted 9 lines
We always do reclaim in stripe unit. Reclaim could create holes in the log, eg, some
io_rangein the middle is reclaimed, butio_rangeat the head remains. So the index list entries don't always have continuous sequence. But this doesn't matter, the firstio_rangeis always the log tail. Superblock has a field pointing to the position of log tail. The hole can waste a lot of disk space though. In the future, we can implement a garbage collection to mitigate the issue, eg, copy data from the index tail to head.
I doubt there would be value in garbage collection. Just flush the old stripes to the RAID. Maybe I'm wrong, but I certainly agree that it isn't a priority.
Quoted 9 lines
In the process reclaim flush data to raid disk, stripe parity will be append to cache log. Parity is always appended after its corresponding data. After parity is in cache disk, a
flush_startblock is appended to log, which indicates stripe is flushing to raid disks. Data writing to raid disks only happens after all data and parity are already in cache disk. This will fix the write whole issue. After a stripe is flushed to raid disks, aflush_endblock is appended to log, which indicates a stripe is settled down in raid disks.
I'm not sure that a separate "flush start" block is needed. Once all the parity blocks have been written we can assume that the flush has started.
A "flush-end" block does make sense, though I would think of it as an 'invalidate' block in that it invalidates specific previous data blocks. Same concept though.
Quoted 16 lines
Recovery relies on the
flush_startandflush_endblock. If recovery finds data and parity of a stripe, the flush start/end block will be used to determine which stage the stripe is in flush. If the stripe is listed in flush end block, the stripe is in raid disks, all data and parity of the stripe can be discarded. If the stripe isn't listed in flush start block, the stripe hasn't started flush to raid yet, its parity can be ignored. Otherwise, the stripe is in the middle of flushing to raid disks. Since we have both data and parity, the recovery just rewrite them to raid disks.IO write code path:
1. copy bio data to stripe memory pages
2. append metadata and data to cache log
3. IO write endio
Quoted 10 lines
reclaim code path:
1. select stripe to reclaim
2. write all stripe data to raid disk
3. in raid5ops_run_io, append metadata and parity data to cache log.ops_run_iodoesn't write data/parity to raid disks at this time
4. flush cache disk and writeflush_startblock
5.ops_run_iocontinues. data/parity will be written to raid disks
6. flush all raid disks cache
7. writeflush_endblock
8. delete in-memory index of the stripe, and advance superblock log checkpoint
I find this a bit confusing. Step 2 writes to the raid disk, but step 3 says it doesn't write to the raid disk yet. It also doesn't tell me when parity is calculated.
I imagine:
1. select stripe to reclaim
2. read missing data or parity block so new parity can be calculated.
3. calculate parity and write to log - this records that the flush has started.
4. flush cache device - probably multiple stripes will get up to the step, and then a single flush will be performed for all of them.
5. schedule writes to the RAID disks, both data an parity.
6. flush RAID disks - again, wait for lots of stripes to reach this point, then perform a single flush
7. write flush_end / invalidate block for all of the flushed stripes
8. delete in-memory index and advance superblock log checkpoint. flush the invalidate blocks before writing the superblock.
If you get '4' for one set of stripes to line up with '8' for the previous set of stripes, then you can combine the two 'flush' operations to the cache devices.
So the cache device see:
FLUSH superblock update, new parity writes, flush_end of older writes FLUSH
which data writes sprinkled through wherever needed.
Quoted 9 lines
Recovery:
Crash in IO write code path doesn't need recovery. If data and checksum don't match, the data will be ignored so read will return old data. In reclaim code path, crash before step 4 doesn't need recovery as data/parity don't touch raid disk yet. Parity can be ignored too. crash after 7 doesn't need recovery too, as the stripe is fully flushed to raid disks. Crash between 4 and 7 need recovery. Data and parity in the log will be written to raid disks.
So recovery involves reading everything in the log, adding data and parity to the stripe cache as it is found, invalidating any entries when an 'invalidate' block is found.
Then any stripe that has all required parity is written to the RAID, and any other stripe is discarded.
Then the log is emptied and we start from scratch.
Quoted 15 lines
The performance of the raid will largely be determined by reclaim speed at run time. Several stages of the reclaim process involves IO wait or disk cache flush, which significantly impact the raid disk utilization and performance. The
flush_startandflush_endblock make running multiple reclaim possible. Each reclaim only records stripes in the flush start/end block which it is reclaiming. Recovery can use the information to correctly determine stripe's flush stage. An example of 2 reclaimer:xxx1 belongs to stripe1 and the same for stripe2
reclaim 1: |data1|...|parity1|...|flush_start1|...|flush_end1| reclaim 2: |data2|..|parity2|...............|flush_start2|...|flush_end2|
Quoted 7 lines
the reclaims only record its own stripes in flush block. If, for exmaple, recovery finds
flush_start1, it knows stripe1 is flushing to raid disk. Recovery will ignore stripe2, since stripe2 isn't inflush_start1.Multiple reclaim will efficiently solve the performance issue. Current code hasn't add multiple reclaim yet, but certainly will be added soon.
Maybe this is what you have in mind, but I would use a state-machine approach for the reclaim
i.e. schedule a collection of stripes for parity calculation.
as they complete, schedule the writes to cache
When there are no pending writes to cache, schedule a flush etc.
Quoted 69 lines
V3:
(quoted diff) +0
-2-fix a use after free bug. fix bio to cache disk crosses disk boundary -adjust memory watermarkV2:
(quoted diff) +0
-3-fix bugs and code clean up -log meta data write isn't FUA any more -discard only runs when the discard area is big enoughSigned-off-by Shaohua Li <[email protected]>--- drivers/md/Makefile | 2 +- drivers/md/raid5-cache.c | 3246 ++++++++++++++++++++++++++++++++++++++++ drivers/md/raid5.c | 69 +- drivers/md/raid5.h | 15 +- include/uapi/linux/raid/md_p.h | 72 + 5 files changed, 3392 insertions(+), 12 deletions(-) create mode 100644 drivers/md/raid5-cache.cdrivers/md/Makefile +1
-1@@ -16,7 +16,7 @@ dm-cache-mq-y += dm-cache-policy-mq.o dm-cache-cleaner-y += dm-cache-policy-cleaner.o dm-era-y += dm-era-target.o md-mod-y += md.o bitmap.o -raid456-y += raid5.o +raid456-y += raid5.o raid5-cache.o # Note: link order is important. All raid personalities # and must come before md.o, as they each initialisedrivers/md/raid5-cache.c new file +31
-0@@ -0,0 +1,3246 @@ +/* +* A caching layer for raid 4/5/6 +* +* Copyright (C) 2015 Shaohua Li <[email protected]> +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program; if not, write to the Free Software Foundation, Inc., +* 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA. +*/ +#include <linux/kernel.h> +#include <linux/wait.h> +#include <linux/blkdev.h> +#include <linux/slab.h> +#include <linux/raid/md_p.h> +#include <linux/crc32.h> +#include <linux/list_sort.h> +#include "md.h" +#include "raid5.h" + +/* the log disk cache will be flushed forcely every second */ +#define LOG_FLUSH_TIME HZ +#define LOG_SUPER_WRITE_TIME (5 * HZ)
drivers/md/raid5-cache.c line 41
Quoted 10 lines
drivers/md/raid5-cache.c new file +10
-0+ +#define MAX_MEM (256 * 1024 * 1024) +#define RECLAIM_BATCH 16 +#define CHECKPOINT_TIMEOUT (5 * HZ) + +#define MAX_RETRY 10 +#define IOERR_RETRY_TIME (5 * 60 * HZ) +#define MEMERR_RETRY_TIME (60 * HZ) + +typedef u64 r5blk_t ;/* log blocks, could be 512B - 4k */
Why isn't this just "sector_t" ??
drivers/md/raid5-cache.c line 44
Quoted 3 lines
drivers/md/raid5-cache.c new file +3
-0+ +struct r5l_log { +struct r5c_cache *cache;
It isn't immediately clear what the "log" is separate from the "cache". If there is a good reason, then a comment with that reason here would be very helpful.
drivers/md/raid5-cache.c line 46
Quoted 2 lines
drivers/md/raid5-cache.c new file +2
-0+struct block_device *bdev; +struct md_rdev *rdev;
Is there a reason for storing both of these instead of just 'rdev' and using 'rdev->bdev' when that is needed?
drivers/md/raid5-cache.c line 59
Quoted 13 lines
drivers/md/raid5-cache.c new file +13
-0+ +struct page *super_page; +u8 uuid[16 ]; +u32 uuid_checksum_data; +u32 uuid_checksum_meta; + +unsigned int block_size;/* bytes */ +unsigned int block_sector_shift; +unsigned int page_block_shift; +unsigned int stripe_data_size;/* sector */ +unsigned int chunk_size;/* sector */ +unsigned int stripe_size;/* sector */ +unsigned int parity_disks;
Some of these are obvious, some are not.
Some more specific comments would help.
Quoted 39 lines
drivers/md/raid5-cache.c new file +39
-0+ +r5blk_t total_blocks; +r5blk_t first_block; +r5blk_t last_block; + +r5blk_t low_watermark;/* For disk space */ +r5blk_t high_watermark; + +r5blk_t last_checkpoint; +r5blk_t last_freepoint; +r5blk_t super_point; +u64 last_cp_seq; +u64 last_freeseq; +unsigned long last_cp_time; +struct work_struct discard_work; + +int do_discard; + +u64 seq;/* get after read log */ +r5blk_t log_start;/* get after read log */ + +u8 data_checksum_type; +u8 meta_checksum_type; + +unsigned int reserved_blocks; +wait_queue_head_t space_waitq; + +struct mutex io_mutex; +struct r5l_io_unit *current_io; + +spinlock_t io_list_lock; +struct list_head running_ios;/* order is important */ + +struct list_head task_list; +struct list_head parity_task_list; +spinlock_t task_lock; + +struct kmem_cache *io_kc; +mempool_t *io_pool;
drivers/md/raid5-cache.c line 108
Quoted 10 lines
drivers/md/raid5-cache.c new file +10
-0+struct bio_set *bio_set; + +unsigned long next_flush_time; +struct work_struct flush_work; +}; + +struct r5l_task ; +/* end function must free task */ +typedef void (r5l_task_end_fn )(struct r5l_task *task,int error); +struct r5l_task {
A comment here telling me what these tasks do would help me understand the data structure.
Quoted 106 lines
drivers/md/raid5-cache.c new file +106
-0+struct list_head list; +int type; +struct bio *bio; +union { +struct bio *orig_bio; +struct { +sector_t stripe_sector; +struct page *page_p; +struct page *page_q; + }; + }; +/* tasks in a single r5l_io_unit will have the same seq and meta_start */ +sector_t meta_start; +sector_t data_start; +u64 seq; +r5l_task_end_fn *fn; +void *private; +unsigned int reserved_blocks; +u32 checksum[]; +}; + +/* +* Data and metadata in log is written with normal IO write. A power failure +* can cause data loss. To relieve the issue, log disk cache will be flushed +* forcely every second. +* +* Note IO is finished out of order. If metadata corrupts in the middle, +* recovery can't work well even metadata/data at the tail is good. IO in log +* tail could finish earlier than IO ahead, so we must be very careful to +* handle FLUSH/FUA bio. +* +* FLUSH bio: the bio will be dispatched after all previous IO finish. FLUSH +* syntax doesn't require pending IO finish, but pending IO might be a metadata +* in the middle of log, we must force this order. +* +* FUA bio: same like FLUSH bio, previous meta IO must be finished before +* dispatching this bio. To simplify implementation, we wait all previous IO. +* And we must add a FLUSH to make sure previous IO hit disk media. metadata +* and the bio itself must be written with FUA. +* */ +struct r5l_io_unit { +struct r5l_log *log; +struct list_head log_sibling; + +struct page *meta_page; +sector_t meta_sector; +int meta_offset; +u64 seq; +struct bio *meta_bio; + +struct list_head tasks; +struct bio *current_bio; +atomic_t refcnt; + +unsigned int has_flush:1 ;/* include flush request */ +unsigned int has_fua:1 ;/* include fua request */ +unsigned int has_null_flush:1 ;/* include empty flush request */ +/* +* io isn't sent yet, flush/fua request can only be submitted till it's +* the first IO in running_ios list +* */ +unsigned int io_deferred:1 ; +int error; +}; + +struct r5c_io_range { +struct list_head log_sibling; +struct list_head stripe_sibling; + +u64 seq; + +sector_t meta_start;/* cache position */ +sector_t data_start;/* cache position */ +sector_t raid_start; +unsigned int data_sectors; + +struct r5c_stripe *stripe; +union { +struct bio *bio; +u32 *checksum;/* only for recovery */ + }; +}; + +struct r5c_stripe { +u64 raid_index; +struct r5c_cache *cache; +atomic_t ref; +int state; +int recovery_state;/* just for recovery */ + +struct list_head io_ranges;/* order list */ +union { +struct list_head stripes; +struct list_head parity_list;/* just for recovery */ + }; + +struct list_head lru; + +int existing_pages; +atomic_t dirty_stripes; +atomic_t pending_bios; +struct page **parity_pages;/* just for recovery */ +struct page *data_pages[]; +}; + +enum {
drivers/md/raid5-cache.c line 224
Quoted 10 lines
drivers/md/raid5-cache.c new file +10
-0+STRIPE_RUNNING =0 , +STRIPE_FROZEN =1 ,/* Doesn't accept new IO */ +STRIPE_PARITY_DONE =2 , +STRIPE_INRAID =3 , +STRIPE_DEAD =4 , + +RECOVERY_NO_FLUSH =0 , +RECOVERY_FLUSH_START =1 ,/* stripe in a start flush block */ + RECOVERY_FLUSH_END = 2, +};
I wouldn't be surprised if some compilers rejected that. Two separate enums would be better.
drivers/md/raid5-cache.c line 232
Quoted 8 lines
drivers/md/raid5-cache.c new file +8
-0+ +#define STRIPE_LOCK_BITS 8 +struct r5c_cache { +struct mddev *mddev; +struct md_rdev *rdev; + +struct r5l_log log; +
Ahh - the log is embedded in the cache.
That probably makes sense. But you don't need a pointer from the log to the cache, just use container_of().
Quoted 56 lines
drivers/md/raid5-cache.c new file +56
-0+spinlock_t tree_lock;/* protect stripe_tree, log_list, full_stripes */ +struct radix_tree_root stripe_tree; +struct list_head log_list;/* sorted list of io_range */ +struct list_head full_stripes; +int full_stripe_cnt; + +struct list_head page_pool; +spinlock_t pool_lock; +u64 free_pages; +u64 total_pages; +u64 max_pages; +u64 low_watermark;/* for memory, pages */ +u64 high_watermark; + +unsigned int stripe_data_size;/* stripe size excluding parity, sector */ +unsigned int chunk_size;/* one disk chunk size including parity, sector */ +unsigned int stripe_size;/* stripe size including parity, sector */ +unsigned int parity_disks; +unsigned int stripe_data_pages; +unsigned int stripe_parity_pages; + +unsigned int reclaim_batch; + +unsigned int reserved_space;/* log reserved size, sector */ + +unsigned long reclaim_reason; +wait_queue_head_t reclaim_wait; +struct md_thread *reclaim_thread; +__le64 *stripe_flush_data; +int quiesce_state; + +int in_recovery; + +struct work_struct pending_io_work; + +spinlock_t stripe_locks[1 <<STRIPE_LOCK_BITS ]; +wait_queue_head_t stripe_waitq[1 <<STRIPE_LOCK_BITS ]; + +int error_state; +int error_type; +int retry_cnt; +unsigned long next_retry_time; +struct bio_list retry_bio_list; +wait_queue_head_t error_wait; + +struct kmem_cache *io_range_kc; +struct kmem_cache *stripe_kc; +struct bio_set *bio_set; +}; + +enum { +RECLAIM_MEM =0 ,/* work hard to reclaim memory */ +RECLAIM_MEM_BACKGROUND =1 ,/* try to reclaim memory */ +RECLAIM_MEM_FULL =2 ,/* only reclaim full stripe */ +RECLAIM_DISK =8 ,/* work hard to reclaim disk */ +RECLAIM_DISK_BACKGROUND =9 ,/* try to reclaim disk */
drivers/md/raid5-cache.c line 298
Quoted 10 lines
drivers/md/raid5-cache.c new file +10
-0+RECLAIM_FLUSH_ALL =16 ,/* flush all data to raid */ + +QUIESCE_NONE =0 , +QUIESCE_START =1 , +QUIESCE_END =2 , + +ERROR_NOERROR =0 , +ERROR_PREPARE =1 ,/* Had an error, flushing cache to raid */ + ERROR_FINISH = 2, /* Had an error, cache has no data */ +};
Three enums here :-)
drivers/md/raid5-cache.c line 300
Quoted 2 lines
drivers/md/raid5-cache.c new file +2
-0+ +#define PAGE_SECTOR_SHIFT (PAGE_SHIFT - 9)
Maybe this should go in md.h - raid1.c raid10.c and raid5.c all use something like it.
... and that's as far as I got.
I really need this a bit more like md-cluster.c: add modules one at a time which I can understand and review and units.
Thanks,
NeilBrown
Re: [PATCH v3 3/8] raid5: A caching layer for RAID5/6
Quoted 10 lines · Neil Brown
On Wed, 3 Jun 2015 15:48:38 -0700 Shaohua Li <[email protected]> wrote:
Hi,
sorry for the delay in getting to this.3000 lines of code is rather hard to review - especially with so few comments :-)
There seem to be a number of fairly well defined modules in the code, such as managing the various data structures. Maybe if these arrived one per patch it would be easier to review them individually.
ok, I'll try to split this one.
Quoted 20 lines
Quoted 20 lines
Main goal of the caching layer is to aggregate IO write to hopefully make full stripe IO write and fix write hole issue. This might speed up read too, but it's not optimized for read, eg, we don't proactively cache data for read. The aggregation makes a lot of sense for workloads which sequentially write to several files with/without fsync. Such workloads are popular in today's datacenter.
Write IO data will write to a cache disk (SSD) first, then later the data will be flushed to raid disks.
The cache disk will be organized as a simple ring buffer log. For IO data, a tuple (
raid_sector, io length, checksum, data) will be appended to the log; for raid 5/6 parity, a tuple (stripe_sector, parity length, checksum, parity data) will be appended to the log. We don't have on-disk index for the data appended to the log. So either we can rebuild an in-memory index at startup with scanning the whole log, or we can flush all data from cache disk to raid disks at shutdown so cache disk has no valid data. Current code chooses the second option, but this can be easily changed.
Quoted 6 lines
Quoted 2 lines
We have a simple meta data for above tuples. It's essentially a tuple (sequence, metadata length).
How are these tuples store in the log? One tuples per block? A block with lots of tuples followed by all the described data? Or does the block of tuples follow the data?
one or several tuples per block. I'll give more details in later post.
Quoted 11 lines
Quoted 6 lines
Crash recovery or startup will scan
the log to read the metadata and rebuild in-memory index. If metadata is broken at the head of the log, even metadata afterward is ok, the scanning will not work well. So we take some steps to mitigate the issue:
-A flush request is sent to cache disk every second to relieve the issueI don't feel at all comfortable about the every-second flush. I can certainly understand a flush after 50% of the log space is written, or even 25%. but time-based flushes should be coming from the filesystem.
Do you like to delete the every-second flush logic? The reclaim will do flush too, so this is barely running, just for the worst case.
Quoted 12 lines
Quoted 2 lines
-FLUSH/FUA request is carefully handled. FLUSH/FUA will only be dispatched after all previous requests finish.
This certainly makes sense.
Quoted 6 lines
The in-memory index is a simple list of
io_range(sequence, metadata sector, data sector, length). The list is orded by sequence. The firstio_rangeentry's metadata sector is the tail of the log. There is also a struct to track io ranges within a stripe. All stripes will be organized as a radix tree.
Quoted 7 lines
It would be useful here to briefly say how these data structures are used.
I assumed the order-by-sequence list is to flush data to RAID when the log is getting full?
The radix-tree tracks where in the log each stripe is - is that correct? So it is easy to find the stripe at the tail of the list to flush it out.
The order-by-sequence list maintains the valid data in log which hasn't been flushed to raid yet. The radix-tree maps stripe sector to stripe.
Quoted 9 lines
Quoted 6 lines
All IO data will be copied to a memory pool for caching too until the data is flushed to raid disks. This is just to simplify the implementation, it's not mandated. In the future flush can do a data copy from cache disk to raid disks, so the memory pool can be discarded. If a stripe is flushed to raid disks, memory of the stripe can be reused.
Again, why a separate memory pool rather than just leaving the data in the stripe cache until it is safe in the RAID?
I'd say a separate pool is much more convenient than using stripe cache. The stripe cache handling state machine is complex enough, which I don't like to make it even more. Plus, the pool isn't a must-have. A small pool will cause frequent stripe flush to raid disks, which impacts both performance and latency. I'm planing to remove it in the future, though if that happens error handling will be hard.
Quoted 7 lines
Quoted 6 lines
We have two limited resources here, memory pool and cache disk space. If resource is tight, we will do reclaim. In either case, we will flush some data from cache disk to raid disks. However, we implement different strategies. For memory reclaim, we prefer reclaiming full stripe. For cache disk space reclaim, we prefer reclaiming
io_rangeentry at the head of index list.
Quoted 7 lines
Wouldn't full stripes be scheduled to the RAID immediately they become full (or immediately after the parity is safe in the log)? So for memory reclaim it would make sense to prefer stripes that have been idle for a long time - so an LRU list.
Certainly when the log approaches full you need to start flushing things at the start of the log.
Yep, a LRU list does make sense, though we don't have it yet. The full stripes are flushed out till we have several such stripes. It's a performance consideration, as flush must flush disk cache, which I don't want to run very frequently.
Quoted 8 lines
Quoted 7 lines
If cache disk has IO error, since all data are in memory pool, we will flush all data to raid disks and fallback to no-cache-disk mode. IO error of cache disk doesn't corrupt any data. After some time, we will try to use cache disk again if the disk is ok. The retry will stop after several rounds of failure.
Quoted 4 lines
The normal data flow involves lots of writing to the cache device and very little if any reading. So we will probably need some sort of "scrub" process to read and verify the log occasionally, just to be sure that reads still work. That could be largely done in user-space.
makes sense.
Quoted 24 lines
Quoted 9 lines
We always do reclaim in stripe unit. Reclaim could create holes in the log, eg, some
io_rangein the middle is reclaimed, butio_rangeat the head remains. So the index list entries don't always have continuous sequence. But this doesn't matter, the firstio_rangeis always the log tail. Superblock has a field pointing to the position of log tail. The hole can waste a lot of disk space though. In the future, we can implement a garbage collection to mitigate the issue, eg, copy data from the index tail to head.I doubt there would be value in garbage collection. Just flush the old stripes to the RAID. Maybe I'm wrong, but I certainly agree that it isn't a priority.
Quoted 9 lines
In the process reclaim flush data to raid disk, stripe parity will be append to cache log. Parity is always appended after its corresponding data. After parity is in cache disk, a
flush_startblock is appended to log, which indicates stripe is flushing to raid disks. Data writing to raid disks only happens after all data and parity are already in cache disk. This will fix the write whole issue. After a stripe is flushed to raid disks, aflush_endblock is appended to log, which indicates a stripe is settled down in raid disks.
Quoted 6 lines
I'm not sure that a separate "flush start" block is needed. Once all the parity blocks have been written we can assume that the flush has started.
A "flush-end" block does make sense, though I would think of it as an 'invalidate' block in that it invalidates specific previous data blocks. Same concept though.
My original implementation doesn't have the 'flush start' block, but when we have multiple reclaim threads (or a state machine you mentioned handling several sets of stripe flushing), the 'flush start' block let recovery code correctly destinguish stripe flush states and finish recovery. The reason is we have multiple set of stripes and crash, recovery can't destinguish a set. For example, a set have 8 stripes writting parity to cache, but recovery only see 7 have parity and it doesn't know there should be 8 stripes. In this case, the parity of 7 stripes should be discarded because disk cache isn't flushed yet (we flush cache disk cache after the 8 stripes have parity in disk), we shouldn't trust the parity data. Maybe the log checksum can help here, I must double check.
Quoted 26 lines
Quoted 15 lines
Recovery relies on the
flush_startandflush_endblock. If recovery finds data and parity of a stripe, the flush start/end block will be used to determine which stage the stripe is in flush. If the stripe is listed in flush end block, the stripe is in raid disks, all data and parity of the stripe can be discarded. If the stripe isn't listed in flush start block, the stripe hasn't started flush to raid yet, its parity can be ignored. Otherwise, the stripe is in the middle of flushing to raid disks. Since we have both data and parity, the recovery just rewrite them to raid disks.IO write code path:
1. copy bio data to stripe memory pages
2. append metadata and data to cache log
3. IO write endioQuoted 10 lines
reclaim code path:
1. select stripe to reclaim
2. write all stripe data to raid disk
3. in raid5ops_run_io, append metadata and parity data to cache log.ops_run_iodoesn't write data/parity to raid disks at this time
4. flush cache disk and writeflush_startblock
5.ops_run_iocontinues. data/parity will be written to raid disks
6. flush all raid disks cache
7. writeflush_endblock
8. delete in-memory index of the stripe, and advance superblock log checkpoint
Quoted 3 lines
I find this a bit confusing. Step 2 writes to the raid disk, but step 3 says it doesn't write to the raid disk yet. It also doesn't tell me when parity is calculated.
It's in step 2, just like step 2 and part of 3 in what you write below
Quoted 15 lines
I imagine:
1. select stripe to reclaim
2. read missing data or parity block so new parity can be calculated.
3. calculate parity and write to log - this records that the flush has started.
4. flush cache device - probably multiple stripes will get up to the step, and then a single flush will be performed for all of them.
5. schedule writes to the RAID disks, both data an parity.
6. flush RAID disks - again, wait for lots of stripes to reach this point, then perform a single flush
7. writeflush_end/ invalidate block for all of the flushed stripes
8. delete in-memory index and advance superblock log checkpoint. flush the invalidate blocks before writing the superblock.
Quoted 6 lines
If you get '4' for one set of stripes to line up with '8' for the previous set of stripes, then you can combine the two 'flush' operations to the cache devices.
So the cache device see:
FLUSH superblock update, new parity writes,flush_endof older writes FLUSH
which data writes sprinkled through wherever needed.
Agree.
Quoted 9 lines
Quoted 8 lines
Recovery:
Crash in IO write code path doesn't need recovery. If data and checksum don't match, the data will be ignored so read will return old data. In reclaim code path, crash before step 4 doesn't need recovery as data/parity don't touch raid disk yet. Parity can be ignored too. crash after 7 doesn't need recovery too, as the stripe is fully flushed to raid disks. Crash between 4 and 7 need recovery. Data and parity in the log will be written to raid disks.
Quoted 6 lines
So recovery involves reading everything in the log, adding data and parity to the stripe cache as it is found, invalidating any entries when an 'invalidate' block is found.
Then any stripe that has all required parity is written to the RAID, and any other stripe is discarded.
Then the log is emptied and we start from scratch.
right
Quoted 23 lines
Quoted 15 lines
The performance of the raid will largely be determined by reclaim speed at run time. Several stages of the reclaim process involves IO wait or disk cache flush, which significantly impact the raid disk utilization and performance. The
flush_startandflush_endblock make running multiple reclaim possible. Each reclaim only records stripes in the flush start/end block which it is reclaiming. Recovery can use the information to correctly determine stripe's flush stage. An example of 2 reclaimer:xxx1 belongs to stripe1 and the same for stripe2
reclaim 1: |data1|...|parity1|...|flush_start1|...|flush_end1| reclaim 2: |data2|..|parity2|...............|flush_start2|...|flush_end2|Quoted 7 lines
the reclaims only record its own stripes in flush block. If, for exmaple, recovery finds
flush_start1, it knows stripe1 is flushing to raid disk. Recovery will ignore stripe2, since stripe2 isn't inflush_start1.Multiple reclaim will efficiently solve the performance issue. Current code hasn't add multiple reclaim yet, but certainly will be added soon.
Quoted 6 lines
Maybe this is what you have in mind, but I would use a state-machine approach for the reclaim
i.e. schedule a collection of stripes for parity calculation.
as they complete, schedule the writes to cache
When there are no pending writes to cache, schedule a flush etc.
the state machine approach is a good choice too. The problem here is reclaim thread sometimes writes cache disk and sometimes write raid disks, the cache disk and raid disks haven't enough IO in the meaning time. With multiple reclaim threads, the threads (hopefully) are in different stage and dispatch enough IO to all disks. a state machien approach can do the similar thing too if it's done correctly. I think we must make sure recover works with multiple stripe flushing sets currently, we can try different reclaim approaches later.
Quoted 73 lines
Quoted 73 lines
V3:
(quoted diff) +0
-2-fix a use after free bug. fix bio to cache disk crosses disk boundary -adjust memory watermarkV2:
(quoted diff) +0
-3-fix bugs and code clean up -log meta data write isn't FUA any more -discard only runs when the discard area is big enoughSigned-off-by Shaohua Li <[email protected]>--- drivers/md/Makefile | 2 +- drivers/md/raid5-cache.c | 3246 ++++++++++++++++++++++++++++++++++++++++ drivers/md/raid5.c | 69 +- drivers/md/raid5.h | 15 +- include/uapi/linux/raid/md_p.h | 72 + 5 files changed, 3392 insertions(+), 12 deletions(-) create mode 100644 drivers/md/raid5-cache.cdrivers/md/Makefile +1
-1@@ -16,7 +16,7 @@ dm-cache-mq-y += dm-cache-policy-mq.o dm-cache-cleaner-y += dm-cache-policy-cleaner.o dm-era-y += dm-era-target.o md-mod-y += md.o bitmap.o -raid456-y += raid5.o +raid456-y += raid5.o raid5-cache.o # Note: link order is important. All raid personalities # and must come before md.o, as they each initialisedrivers/md/raid5-cache.c new file +35
-0@@ -0,0 +1,3246 @@ +/* +* A caching layer for raid 4/5/6 +* +* Copyright (C) 2015 Shaohua Li <[email protected]> +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program; if not, write to the Free Software Foundation, Inc., +* 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA. +*/ +#include <linux/kernel.h> +#include <linux/wait.h> +#include <linux/blkdev.h> +#include <linux/slab.h> +#include <linux/raid/md_p.h> +#include <linux/crc32.h> +#include <linux/list_sort.h> +#include "md.h" +#include "raid5.h" + +/* the log disk cache will be flushed forcely every second */ +#define LOG_FLUSH_TIME HZ +#define LOG_SUPER_WRITE_TIME (5 * HZ) + +#define MAX_MEM (256 * 1024 * 1024) +#define RECLAIM_BATCH 16 +#define CHECKPOINT_TIMEOUT (5 * HZ)
Quoted 8 lines
drivers/md/raid5-cache.c line 41
Quoted 6 lines
drivers/md/raid5-cache.c new file +6
-0+ +#define MAX_RETRY 10 +#define IOERR_RETRY_TIME (5 * 60 * HZ) +#define MEMERR_RETRY_TIME (60 * HZ) + +typedef u64 r5blk_t ;/* log blocks, could be 512B - 4k */ Why isn't this just "
sector_t" ??
It can be sector_t. But with the new type, it's clear to know code is working on a sector or a block. It helps avoid bug.
Quoted 7 lines
drivers/md/raid5-cache.c line 44
Quoted 3 lines
drivers/md/raid5-cache.c new file +3
-0+ +struct r5l_log { +struct r5c_cache *cache;It isn't immediately clear what the "log" is separate from the "cache". If there is a good reason, then a comment with that reason here would be very helpful.
ok, will do
Quoted 5 lines
drivers/md/raid5-cache.c line 46
Quoted 2 lines
drivers/md/raid5-cache.c new file +2
-0+struct block_device *bdev; +struct md_rdev *rdev;Is there a reason for storing both of these instead of just 'rdev' and using '
rdev->bdev' when that is needed?
It's just for convenience
Quoted 17 lines
Quoted 4 lines
drivers/md/raid5-cache.c new file +3
-0+ +struct page *super_page; +u8 uuid[16 ];drivers/md/raid5-cache.c line 59
Quoted 10 lines
drivers/md/raid5-cache.c new file +10
-0+u32 uuid_checksum_data; +u32 uuid_checksum_meta; + +unsigned int block_size;/* bytes */ +unsigned int block_sector_shift; +unsigned int page_block_shift; +unsigned int stripe_data_size;/* sector */ +unsigned int chunk_size;/* sector */ +unsigned int stripe_size;/* sector */ +unsigned int parity_disks;Some of these are obvious, some are not.
Some more specific comments would help.
Ok, I'll split the patch and add more comments.
Thanks,
Shaohua