bcache: improve large bucket size on-disk layout
Subsystems (3)
cover bcache: improve large bucket size on-disk layout
Current large bucket (size >16MB) layout is a preparation for the future zoned cache device, by which the cache device bucket can be aligned to zone size and boundary of the zoned SSD.
Commit ffa470327572 ("bcache: add bucket_size_hi into struct cache_sb_disk for large bucket") extends bucket size from 16MB to 1TB, but this change is problematic and can be improved,
- The bucket size is always power-of-2 value, it is unnecessary to add extra member in struct cache_sb_disk to store the real size value. Just using existing bucket_size to store the order of power-of-2 value is enough.
- The added bucket_size_hi is behind d[SB_JOURNAL_BUCKETS] in struct cache_sb_disk, which breaks a very implicit restriction from macro csum_set(). This restriction requires member d[] must be the last member in data structure to calculate the checksum.
This series is to improve the above issue from bcache kernel code. The basic ideas of this series are,
- Add BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE incompat feature bit. When it is set, for bucket size >16MB, bucket_size stores the order of power-of-2 bucket size value. Then no extra space introduced and csum_set() has all data to calculate super block checksum.
- Rename struct cache_sb_disk's bucket_size_hi to obso_bucket_size_hi, and rename the incompat feature bit BCH_FEATURE_INCOMPAT_LARGE_BUCKET to BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET, to indicate they are not used anymore and obsoleted.
- If a cache device was created for bucket size >16MB with obsoleted layout, bcache driver will print error message for the obsoleted layout and all following attached bcache device will be read-only. If there is dirty data on cache device, after the writeback accomplished the cache device can be re-created with new bucket_size usage by the latest bcache-tools.
Because the btree node usage is not optimized, I doubt there is people really uses bucket size >16MB at this moment, except me. But this is something needs to be improved and better as soon as possible.
Coly Li
---
Coly Li (4):
bcache: fix typo from SUUP to SUPP in features.h
bcache: check unsupported feature sets for bcache register
bcache: introduce BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE for large
bucket
bcache: set bcache device into read-only mode for
BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET
drivers/md/bcache/features.c | 2 +- drivers/md/bcache/features.h | 30 +++++++++++++++++---- drivers/md/bcache/super.c | 51 +++++++++++++++++++++++++++++++++--- include/uapi/linux/bcache.h | 2 +- 4 files changed, 75 insertions(+), 10 deletions(-)
Signature · 1 line
2.26.2
PATCH 1/4 bcache: fix typo from SUUP to SUPP in features.h
This patch fixes the following typos,
from BCH_FEATURE_COMPAT_SUUP to BCH_FEATURE_COMPAT_SUPP from BCH_FEATURE_INCOMPAT_SUUP to BCH_FEATURE_INCOMPAT_SUPP from BCH_FEATURE_INCOMPAT_SUUP to BCH_FEATURE_RO_COMPAT_SUPP
--- drivers/md/bcache/features.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/md/bcache/features.h +3 -3
@@ -15,9 +15,9 @@/* Incompat feature set */ #define BCH_FEATURE_INCOMPAT_LARGE_BUCKET 0x0001/* 32bit bucket size */ -#define BCH_FEATURE_COMPAT_SUUP 0 -#define BCH_FEATURE_RO_COMPAT_SUUP 0 -#define BCH_FEATURE_INCOMPAT_SUUP BCH_FEATURE_INCOMPAT_LARGE_BUCKET +#define BCH_FEATURE_COMPAT_SUPP 0 +#define BCH_FEATURE_RO_COMPAT_SUPP 0 +#define BCH_FEATURE_INCOMPAT_SUPP BCH_FEATURE_INCOMPAT_LARGE_BUCKET#define BCH_HAS_COMPAT_FEATURE (sb, mask) \ ((sb)->feature_compat & (mask))
Signature · 1 line
2.26.2
PATCH 2/4 bcache: check unsupported feature sets for bcache register
In reply to Coly LiThis patch adds the check for features which is incompatible for current supported feature sets.
Now if the bcache device created by bcache-tools has features that current kernel doesn't support, read_super() will fail with error messoage. E.g. if an unsupported incompatible feature detected, bcache register will fail with dmesg "bcache: register_bcache() error :
Unsupported incompatible feature found".
--- drivers/md/bcache/features.h | 15 +++++++++++++++ drivers/md/bcache/super.c | 14 ++++++++++++++ 2 files changed, 29 insertions(+)
drivers/md/bcache/features.h +15 -0
@@ -79,6 +79,21 @@ static inline void bch_clear_feature_##name(struct cache_sb *sb) \BCH_FEATURE_INCOMPAT_FUNCS (large_bucket,LARGE_BUCKET ); +static inline bool bch_has_unknown_compat_features (struct cache_sb *sb) +{ +return ((sb->feature_compat & ~BCH_FEATURE_COMPAT_SUPP ) !=0 ); +} + +static inline bool bch_has_unknown_ro_compat_features (struct cache_sb *sb) +{ +return ((sb->feature_ro_compat & ~BCH_FEATURE_RO_COMPAT_SUPP ) !=0 ); +} + +static inline bool bch_has_unknown_incompat_features (struct cache_sb *sb) +{ +return ((sb->feature_incompat & ~BCH_FEATURE_INCOMPAT_SUPP ) !=0 ); +} +int bch_print_cache_set_feature_compat (struct cache_set *c,char *buf,int size);int bch_print_cache_set_feature_ro_compat (struct cache_set *c,char *buf,int size);int bch_print_cache_set_feature_incompat (struct cache_set *c,char *buf,int size);
drivers/md/bcache/super.c +14 -0
@@ -228,6 +228,20 @@ static const char *read_super(struct cache_sb *sb, struct block_device *bdev, sb->feature_compat =le64_to_cpu (s->feature_compat); sb->feature_incompat =le64_to_cpu (s->feature_incompat); sb->feature_ro_compat =le64_to_cpu (s->feature_ro_compat); + +/* Check incompatible features */ + err ="Unsupported compatible feature found" ; +if (bch_has_unknown_compat_features (sb)) + goto err; + + err ="Unsupported read-only compatible feature found" ; +if (bch_has_unknown_ro_compat_features (sb)) + goto err; + + err ="Unsupported incompatible feature found" ; +if (bch_has_unknown_incompat_features (sb)) + goto err; + err =read_super_common (sb, bdev, s);if (err) goto err;
Signature · 1 line
2.26.2
PATCH 3/4 bcache: introduce BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE for large bucket
In reply to Coly LiWhen large bucket feature was added, BCH_FEATURE_INCOMPAT_LARGE_BUCKET was introduced into the incompat feature set. It used bucket_size_hi (which was added at the tail of struct cache_sb_disk) to extend current 16bit bucket size to 32bit with existing bucket_size in struct cache_sb_disk.
This is not a good idea, there are two obvious problems,
- Bucket size is always value power of 2, if store log2(bucket size) in existing bucket_size of struct cache_sb_disk, it is unnecessary to add bucket_size_hi.
- Macro csum_set() assumes d[SB_JOURNAL_BUCKETS] is the last member in struct cache_sb_disk, bucket_size_hi was added after d[] which makes csum_set calculate an unexpected super block checksum.
To fix the above problems, this patch introduces a new incompat feature bit BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE, when this bit is set, it means bucket_size in struct cache_sb_disk stores the order of power-of-2 bucket size value. When user specifies a bucket size larger than 32768 sectors, BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE will be set to incompat feature set, and bucket_size stores log2(bucket size) more than store the real bucket size value.
The obsoleted BCH_FEATURE_INCOMPAT_LARGE_BUCKET won't be used anymore, it is renamed to BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET and still only recognized by kernel driver for legacy compatible purpose. The previous bucket_size_hi is renmaed to obso_bucket_size_hi in struct cache_sb_disk and not used in bcache-tools anymore.
For cache device created with BCH_FEATURE_INCOMPAT_LARGE_BUCKET feature, bcache-tools and kernel driver still recognize the feature string and display it as "obso_large_bucket".
With this change, the unnecessary extra space extend of bcache on-disk super block can be avoided, and csum_set() may generate expected check sum as well.
--- drivers/md/bcache/features.c | 2 +- drivers/md/bcache/features.h | 11 ++++++++--- drivers/md/bcache/super.c | 22 +++++++++++++++++++--- include/uapi/linux/bcache.h | 2 +- 4 files changed, 29 insertions(+), 8 deletions(-)
drivers/md/bcache/features.c +1 -1
@@ -17,7 +17,7 @@ struct feature { };static struct feature feature_list[] = { - {BCH_FEATURE_INCOMPAT ,BCH_FEATURE_INCOMPAT_LARGE_BUCKET , + {BCH_FEATURE_INCOMPAT ,BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE ,"large_bucket" }, {0 ,0 ,0 }, };
drivers/md/bcache/features.h +8 -3
@@ -13,11 +13,15 @@/* Feature set definition */ /* Incompat feature set */ -#define BCH_FEATURE_INCOMPAT_LARGE_BUCKET 0x0001/* 32bit bucket size */ +/* 32bit bucket size, obsoleted */ +#define BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET 0x0001 +/* real bucket size is (1 << bucket_size) */ +#define BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE 0x0002#define BCH_FEATURE_COMPAT_SUPP 0#define BCH_FEATURE_RO_COMPAT_SUPP 0 -#define BCH_FEATURE_INCOMPAT_SUPP BCH_FEATURE_INCOMPAT_LARGE_BUCKET +#define BCH_FEATURE_INCOMPAT_SUPP (BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET| \ + BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE)#define BCH_HAS_COMPAT_FEATURE (sb, mask) \ ((sb)->feature_compat & (mask))
@@ -77,7 +81,8 @@ static inline void bch_clear_feature_##name(struct cache_sb *sb) \ ~BCH##_FEATURE_INCOMPAT_##flagname; \ } -BCH_FEATURE_INCOMPAT_FUNCS (large_bucket,LARGE_BUCKET ); +BCH_FEATURE_INCOMPAT_FUNCS (obso_large_bucket,OBSO_LARGE_BUCKET ); +BCH_FEATURE_INCOMPAT_FUNCS (large_bucket,LOG_LARGE_BUCKET_SIZE );static inline bool bch_has_unknown_compat_features (struct cache_sb *sb) {
drivers/md/bcache/super.c +19 -3
@@ -64,9 +64,25 @@ static unsigned int get_bucket_size(struct cache_sb *sb, struct cache_sb_disk *s {unsigned int bucket_size =le16_to_cpu (s->bucket_size); -if (sb->version >=BCACHE_SB_VERSION_CDEV_WITH_FEATURES && -bch_has_feature_large_bucket (sb)) - bucket_size |=le16_to_cpu (s->bucket_size_hi) <<16 ; +if (sb->version >=BCACHE_SB_VERSION_CDEV_WITH_FEATURES ) { +if (bch_has_feature_large_bucket (sb)) { +unsigned int max, order; + + max =sizeof (unsigned int ) *BITS_PER_BYTE -1 ; + order =le16_to_cpu (s->bucket_size); +/* +* bcache tool will make sure the overflow won't +* happen, an error message here is enough. +*/ +if (order > max) +pr_err ("Bucket size (1 << %u) overflows\n" , + order); + bucket_size =1 << order; + }else if (bch_has_feature_obso_large_bucket (sb)) { + bucket_size += +le16_to_cpu (s->obso_bucket_size_hi) <<16 ; + } + }return bucket_size; }
include/uapi/linux/bcache.h +1 -1
@@ -213,7 +213,7 @@ struct cache_sb_disk {__le16 keys; };__le64 d[SB_JOURNAL_BUCKETS ];/* journal buckets */ -__le16 bucket_size_hi; +__le16 obso_bucket_size_hi;/* obsoleted */ };/*
Signature · 1 line
2.26.2
PATCH 4/4 bcache: set bcache device into read-only mode for BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET
In reply to Coly LiIf BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET is set in incompat feature set, it means the cache device is created with obsoleted layout with obso_bucket_site_hi. Now bcache does not support this feature bit, a new BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE incompat feature bit is added for a better layout to support large bucket size.
For the legacy compatibility purpose, if a cache device created with obsoleted BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET feature bit, all bcache devices attached to this cache set should be set to read-only. Then the dirty data can be written back to backing device before re-create the cache device with BCH_FEATURE_INCOMPAT_LOG_LARGE_BUCKET_SIZE feature bit by the latest bcache-tools.
This patch checks BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET feature bit when running a cache set and attach a bcache device to the cache set. If this bit is set,
- When run a cache set, print an error kernel message to indicate all following attached bcache device will be read-only.
- When attach a bcache device, print an error kernel message to indicate the attached bcache device will be read-only, and ask users to update to latest bcache-tools.
Such change is only for cache device whose bucket size >= 32MB, this is for the zoned SSD and almost nobody uses such large bucket size at this moment. If you don't explicit set a large bucket size for a zoned SSD, such change is totally transparent to your bcache device.
--- drivers/md/bcache/super.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/md/bcache/super.c +15 -0
@@ -1332,6 +1332,12 @@ int bch_cached_dev_attach(struct cached_dev *dc, struct cache_set *c,bcache_device_link (&dc->disk, c,"bdev" );atomic_inc (&c->attached_dev_nr); +if (bch_has_feature_obso_large_bucket (&(c->cache->sb))) { +pr_err ("The obsoleted large bucket layout is unsupported, set the bcache device into read-only\n" ); +pr_err ("Please update to the latest bcache-tools to create the cache device\n" ); +set_disk_ro (dc->disk.disk,1 ); + } +/* Allow the writeback thread to proceed */ up_write (&dc->writeback_lock);
@@ -1554,6 +1560,12 @@ static int flash_dev_run(struct cache_set *c, struct uuid_entry *u)bcache_device_link (d, c,"volume" ); +if (bch_has_feature_obso_large_bucket (&c->cache->sb)) { +pr_err ("The obsoleted large bucket layout is unsupported, set the bcache device into read-only\n" ); +pr_err ("Please update to the latest bcache-tools to create the cache device\n" ); +set_disk_ro (d->disk,1 ); + } +return 0 ; err:kobject_put (&d->kobj);
@@ -2113,6 +2125,9 @@ static int run_cache_set(struct cache_set *c) c->cache->sb.last_mount = (u32 )ktime_get_real_seconds ();bcache_write_super (c); +if (bch_has_feature_obso_large_bucket (&c->cache->sb)) +pr_err ("Detect obsoleted large bucket layout, all attached bcache device will be read-only\n" ); +list_for_each_entry_safe (dc, t, &uncached_devices, list)bch_cached_dev_attach (dc, c,NULL );
Signature · 1 line
2.26.2