btrfs: reduce the memory usage for replace in btrfs_io_context.
cover btrfs: reduce the memory usage for replace in btrfs_io_context.
In btrfs_io_context, we have two members dedicated for dev-replace:
- num_tgtdevs
This is straight-forward, just the number of extra stripes for replace usage.
- tgtdev_map[]
This is a little complex, it represents the mapping between the original stripes and dev-replace stripes.
This is mostly for RAID56, as only in RAID56 the stripes contain different contents, thus it's important to know the mapping.
It goes like this:
num_stripes = 4 (3 + 1 for replace)
stripes[0]: dev = devid 1, physical = X
stripes[1]: dev = devid 2, physical = Y
stripes[2]: dev = devid 3, physical = Z
stripes[3]: dev = devid 0, physical = Y
num_tgtdevs = 1
tgtdev_map[0] = 0 <- Means stripes[0] is not involved in replace.
tgtdev_map[1] = 3 <- Means stripes[1] is involved in replace,
and it's duplicated to stripes[3].
tgtdev_map[2] = 0 <- Means stripes[2] is not involved in replace.
Thus most space is wasted, and the more devices in the array, the more space wasted.
For the current tgtdev_map[] design, it's wasting quite some space. E.g. in the above case, we only need on slot to record the source stripe number, and the other two slots are just a waste of space.
The existing tgtdev_map[] will make more sense if we support multiple running dev-replaces, but that's not the case.
So this patch would mostly change it to a new, and more space efficient way, by going something like this for the same example:
replace_nr_stripes = 1
tgtdev_map[0] = 1 <- Means stripes[1] is involved in replace.
tgtdev_map[1] = -1 <- Means the second slot is not used.
(Only DUP can use this slot, but they
don't really care)
Furthermore we reduce the width of nr_stripes related member to u16, the same as on-disk format width.
This not only saved some space for btrfs_io_context structure, but also allows the following cleanups:
- Streamline handle_ops_on_dev_replace()
We go a common path for both WRITE and GET_READ_MIRRORS, and only for DUP and GET_READ_MIRRORS, we shrink the bioc to keep the same old behavior.
- Remove some unnecessary variables
Although the series still increases the number of lines, the net increase mostly comes from comments, in fact around 70 lines of comments are added around the replace related members.
Qu Wenruo (3):
btrfs: simplify the @bioc argument for handle_ops_on_dev_replace()
btrfs: small improvement for btrfs_io_context structure
btrfs: use a more space efficient way to represent the source of
duplicated stripes
fs/btrfs/raid56.c | 44 +++++++++-- fs/btrfs/scrub.c | 4 +- fs/btrfs/volumes.c | 187 +++++++++++++++++++++------------------------ fs/btrfs/volumes.h | 52 +++++++++++-- 4 files changed, 174 insertions(+), 113 deletions(-)
Signature · 1 line
2.39.1
PATCH 1/3 btrfs: simplify the @bioc argument for handle_ops_on_dev_replace()
There is no memory re-allocation for handle_ops_on_dev_replace(), thus we don't need to pass a struct btrfs_io_context ** pointer.
Just a regular struct btrfs_io_contex * pointer is enough.
--- fs/btrfs/volumes.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
fs/btrfs/volumes.c +2 -4
@@ -6168,12 +6168,11 @@ static bool is_block_group_to_copy(struct btrfs_fs_info *fs_info, u64 logical) }static void handle_ops_on_dev_replace (enum btrfs_map_op op, -struct btrfs_io_context **bioc_ret, +struct btrfs_io_context *bioc,struct btrfs_dev_replace *dev_replace,u64 logical,int *num_stripes_ret,int *max_errors_ret) { -struct btrfs_io_context *bioc = *bioc_ret;u64 srcdev_devid = dev_replace->srcdev->devid;int tgtdev_indexes =0 ;int num_stripes = *num_stripes_ret;
@@ -6262,7 +6261,6 @@ static void handle_ops_on_dev_replace(enum btrfs_map_op op, *num_stripes_ret = num_stripes; *max_errors_ret = max_errors; bioc->num_tgtdevs = tgtdev_indexes; - *bioc_ret = bioc; }static bool need_full_stripe (enum btrfs_map_op op)
@@ -6604,7 +6602,7 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,if (dev_replace_is_ongoing && dev_replace->tgtdev !=NULL &&need_full_stripe (op)) { -handle_ops_on_dev_replace (op, &bioc, dev_replace, logical, +handle_ops_on_dev_replace (op, bioc, dev_replace, logical, &num_stripes, &max_errors); }
Signature · 1 line
2.39.1
Re: [PATCH 1/3] btrfs: simplify the @bioc argument for handle_ops_on_dev_replace()
Looks good,
Re: [PATCH 1/3] btrfs: simplify the @bioc argument for handle_ops_on_dev_replace()
In reply to Qu WenruoLGTM.
PATCH 2/3 btrfs: small improvement for btrfs_io_context structure
In reply to Qu WenruoThat structure is our ultimate objective for all __btrfs_map_block() related functions.
We have some hard to understand members, like tgtdev_map, but without any comments.
This patch will improve the situation by:
- Add extra comments for num_stripes, mirror_num, num_tgtdevs and tgtdev_map[]
Especially for the last two members, add a dedicated (thus very long) comments for them, with example to explain it.
- Shrink those int members to u16.
In fact our on-disk format is only using u16 for num_stripes, thus no need to go int at all.
- Add extra ASSERT() for alloc_btrfs_io_context() for the stripes argument
--- fs/btrfs/volumes.c | 15 ++++++++++--- fs/btrfs/volumes.h | 53 +++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 60 insertions(+), 8 deletions(-)
fs/btrfs/volumes.c +12 -3
@@ -5888,13 +5888,22 @@ static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_ int total_stripes, int real_stripes) { - struct btrfs_io_context *bioc = kzalloc( +struct btrfs_io_context *bioc; + +/* +* We should have valid number of stripes, larger than U16_MAX(65535) +* indicates something totally wrong, as our on-disk format is only +* at u16. +*/ +ASSERT (total_stripes <U16_MAX && total_stripes >0 ); + + bioc = kzalloc( /* The size of btrfs_io_context */ sizeof(struct btrfs_io_context) + /* Plus the variable array for the stripes */ sizeof(struct btrfs_io_stripe) * (total_stripes) + /* Plus the variable array for the tgt dev */ - sizeof(int) * (real_stripes) + + sizeof(u16) * (real_stripes) +/* * Plus the raid_map, which includes both the tgt dev * and the stripes.
@@ -5908,7 +5917,7 @@ static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_refcount_set (&bioc->refs,1 ); bioc->fs_info = fs_info; - bioc->tgtdev_map = (int *)(bioc->stripes + total_stripes); + bioc->tgtdev_map = (u16 *)(bioc->stripes + total_stripes); bioc->raid_map = (u64 *)(bioc->tgtdev_map + real_stripes);return bioc;
fs/btrfs/volumes.h +48 -5
59 lines not on this page · raw message
Signature · 1 line
2.39.1
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
fs/btrfs/volumes.c static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_ · line 5898
Quoted 13 lines · Qu Wenruo
fs/btrfs/volumes.c +8
-1@@ -5888,13 +5888,22 @@ static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_ int total_stripes, int real_stripes) { - struct btrfs_io_context *bioc = kzalloc( +struct btrfs_io_context *bioc; + +/* +* We should have valid number of stripes, larger than U16_MAX(65535) +* indicates something totally wrong, as our on-disk format is only +* at u16. +*/ +ASSERT (total_stripes <U16_MAX && total_stripes >0 );
Shouldn't we better change 'int total_stripes' and 'int real_stripes' to u16? That'll imply above ASSERT()
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
Quoted 17 lines · Johannes Thumshirn
fs/btrfs/volumes.c static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_ · line 5898
Quoted 13 lines · Qu Wenruo
fs/btrfs/volumes.c +8
-1@@ -5888,13 +5888,22 @@ static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_ int total_stripes, int real_stripes) { - struct btrfs_io_context *bioc = kzalloc( +struct btrfs_io_context *bioc; + +/* +* We should have valid number of stripes, larger than U16_MAX(65535) +* indicates something totally wrong, as our on-disk format is only +* at u16. +*/ +ASSERT (total_stripes <U16_MAX && total_stripes >0 );Shouldn't we better change 'int
total_stripes' and 'intreal_stripes' to u16? That'll imply aboveASSERT()
Right, that is way better.
Thanks,
Qu
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
In reply to Qu Wenruofs/btrfs/volumes.c static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_ · line 5909
Quoted 13 lines
fs/btrfs/volumes.c +2
-1+ bioc = kzalloc( /* The size of btrfs_io_context */ sizeof(struct btrfs_io_context) + /* Plus the variable array for the stripes */ sizeof(struct btrfs_io_stripe) * (total_stripes) + /* Plus the variable array for the tgt dev */ -sizeof (int ) * (real_stripes) + + sizeof(u16) * (real_stripes) +/* * Plus the raid_map, which includes both the tgt dev * and the stripes.
Why not order the various sizeof() in the same manner as in the struct btrfs_io_context?
Thx.
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
Quoted 14 lines · Anand Jain
Quoted 13 lines
+ bioc = kzalloc(
/* The size ofbtrfs_io_context*/sizeof(struct btrfs_io_context)+
/* Plus the variable array for the stripes */sizeof(struct btrfs_io_stripe)* (total_stripes) + /* Plus the variable array for the tgt dev */ - sizeof(int) * (real_stripes) +
+ sizeof(u16) * (real_stripes) +
/*
* Plus theraid_map, which includes both the tgt dev
* and the stripes.
Quoted 1 line
Why not order the various
sizeof()in the same manner as in the structbtrfs_io_context?
Because the tgtdev_map would soon get completely removed in the next patch.
Just to mention, I don't like the current way to allocate memory at all.
If there is more feedback, I can convert the allocation to the same way as alloc_rbio() of raid56.c, AKA, use dedicated kcalloc() calls for those arrays.
Thanks,
Qu
Quoted 2 lines
Thx.
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
Quoted 17 lines · Qu Wenruo
Quoted 14 lines · Anand Jain
Quoted 13 lines
+ bioc = kzalloc(
/* The size ofbtrfs_io_context*/sizeof(struct btrfs_io_context)+
/* Plus the variable array for the stripes */sizeof(struct btrfs_io_stripe)* (total_stripes) + /* Plus the variable array for the tgt dev */ - sizeof(int) * (real_stripes) +
+ sizeof(u16) * (real_stripes) +
/*
* Plus theraid_map, which includes both the tgt dev
* and the stripes.
Quoted 4 lines
Quoted 1 line
Why not order the various
sizeof()in the same manner as in the structbtrfs_io_context?Because the
tgtdev_mapwould soon get completely removed in the next patch.
Ah. Ok.
Quoted 3 lines
Just to mention, I don't like the current way to allocate memory at all.
If there is more feedback, I can convert the allocation to the same way as
alloc_rbio()ofraid56.c, AKA, use dedicatedkcalloc()calls for those arrays.
The alloc_rbio() method allocates each component independently, leading
to frequent memory allocation and freeing, which could impact performance in an IO context.
It may be goodidea to conduct a small experiment to assess any performance penalties. If none are detected, then it should be ok.
Thx,
Quoted 6 lines
Thanks,
QuQuoted 2 lines
Thx.
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
Quoted 25 lines · Anand Jain
Quoted 17 lines · Qu Wenruo
Quoted 14 lines · Anand Jain
Quoted 13 lines
+ bioc = kzalloc(
/* The size ofbtrfs_io_context*/sizeof(struct btrfs_io_context)+
/* Plus the variable array for the stripes */sizeof(struct btrfs_io_stripe)* (total_stripes) + /* Plus the variable array for the tgt dev */ - sizeof(int) * (real_stripes) +
+ sizeof(u16) * (real_stripes) +
/*
* Plus theraid_map, which includes both the tgt dev
* and the stripes.Quoted 4 lines
Quoted 1 line
Why not order the various
sizeof()in the same manner as in the structbtrfs_io_context?Because the
tgtdev_mapwould soon get completely removed in the next patch.
Quoted 8 lines
Ah. Ok.
Quoted 3 lines
Just to mention, I don't like the current way to allocate memory at all.
If there is more feedback, I can convert the allocation to the same way as
alloc_rbio()ofraid56.c, AKA, use dedicatedkcalloc()calls for those arrays.The
alloc_rbio()method allocates each component independently, leading
to frequent memory allocation and freeing, which could impact performance in an IO context.
That method is fine if we only have one variable length array (stripes).
But it won't be a problem anymore, the 3/3 would make the dev-replace to only take one s16 to record the replace source.
Then I have a new patch to this series, which converts *raid_map into a single u64.
By all those work, btrfs_io_context structure would only have one variable length array (stripes), then the existing method would be acceptable.
The final result would make btrfs_io_context look like this: (comments skipped, as the comments would be larger than the structure itself)
struct btrfs_io_context {
refcount_t refs;
struct btrfs_fs_info *fs_info;
u64 map_type; /* get from map_lookup->type */
struct bio *orig_bio;
atomic_t error;
u16 max_errors;
u16 num_stripes;
u16 mirror_num;
u16 replace_nr_stripes;
s16 replace_stripe_src;
u64 full_stripe_logical;
struct btrfs_io_stripe stripes[];
};Quoted 3 lines
It may be goodidea to conduct a small experiment to assess any performance penalties. If none are detected, then it should be ok.
If HCH or someone else is going to make btrfs_io_context mempool based, then I guess we have to go the other way, mempool for btrfs_io_context, but still manually allocate the stripes.
Anyway the updated series would benefit everyone (at least I hope so).
Thanks,
Qu
Quoted 10 lines
Thx,
Quoted 6 lines
Thanks,
QuQuoted 2 lines
Thx.
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
In reply to Qu WenruoQuoted 7 lines · Qu Wenruo
Because the
tgtdev_mapwould soon get completely removed in the next patch.Just to mention, I don't like the current way to allocate memory at all.
If there is more feedback, I can convert the allocation to the same way as
alloc_rbio()ofraid56.c, AKA, use dedicatedkcalloc()calls for those arrays.
The real elephant in the room is that the io_context needs to use a mempool anyway to be deadlock safe. Which means we'll need to size for the worst case (per file system?) here. Fortunately the read fast path now doesn't use the io_context at all which helps with the different sizing optimizations.
Re: [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure
Quoted 11 lines · Christoph Hellwig
Quoted 7 lines · Qu Wenruo
Because the
tgtdev_mapwould soon get completely removed in the next patch.Just to mention, I don't like the current way to allocate memory at all.
If there is more feedback, I can convert the allocation to the same way as
alloc_rbio()ofraid56.c, AKA, use dedicatedkcalloc()calls for those arrays.The real elephant in the room is that the
io_contextneeds to use a mempool anyway to be deadlock safe.
In fact, that makes more sense to migrate to the raid56 method.
We go mempool for the io_context structure itself, then still do regular memory allocation for the stripes (and the raid56 map).
Furthermore we can skip the raid56_map if we don't need, especially currently we're wasting memory for non-raid56 profiles.
And with my current work to make that tgtdev_map static (currently two u16, but next version it would be just one u16), for non-raid56 cases we will just do one mempool alloc (io_context) + one regular alloc (stripes).
Which I guess won't be that bad compared to the existing one.
Thanks,
Qu
Quoted 4 lines
Which means we'll need to size
for the worst case (per file system?) here. Fortunately the read fast path now doesn't use theio_contextat all which helps with the different sizing optimizations.
PATCH 3/3 btrfs: use a more space efficient way to represent the source of duplicated stripes
In reply to Qu WenruoFor btrfs dev-replace, we have to duplicate writes to the source device into the target device.
For non-RAID56, all writes into the same mapped ranges are sharing the same content, thus they don't really need to bother anything. (E.g. in btrfs_submit_bio() for non-RAID56 range we just submit the
same write to all involved devices).
But for RAID56, all stripes contain different content, thus we must have a clear mapping of which stripe is duplicated from which original stripe.
Currently we use a complex way using tgtdev_map[] array, e.g:
num_tgtdevs = 1
tgtdev_map[0] = 0 <- Means stripes[0] is not involved in replace.
tgtdev_map[1] = 3 <- Means stripes[1] is involved in replace,
and it's duplicated to stripes[3].
tgtdev_map[2] = 0 <- Means stripes[2] is not involved in replace.
But this is wasting some space, and ignored one important thing for dev-replace, there is at most ONE running replace.
Thus we can change it to a fixed array to represent the mapping:
replace_nr_stripes = 1
tgtdev_map[0] = 1 <- Means stripes[1] is involved in replace.
tgtdev_map[1] = -1 <- Means the second slot is not used.
(Only DUP can use this slot, but they
don't really care)
By this we can save some space for large RAID56 chunks.
Thus the patch involves the following changes:
- Replace @num_tgtdevs and @tgtdev_map[] with @replace_nr_stripes and @replace_stripe_src[2]
@num_tgtdevs is just renamed to @replace_nr_stripes. While the mapping is completely changed.
- Add extra ASSERT()s for RAID56 code
- Only add two more extra stripes for dev-replace cases.
As we have a upper limit on how many dev-replace stripes we can have.
- Unify the behavior of handle_ops_on_dev_replace() Previously handle_ops_on_dev_replace() go two different paths for WRITE and GET_READ_MIRRORS.
Now unify them by always go WRITE path first (with at most 2 replace stripes), then if we're doing GET_READ_MIRRORS and we have 2 extra stripes, just drop one stripe.
- Remove the @real_stripes argument from alloc_btrfs_io_context() As we don't need the old variable length array any more.
--- fs/btrfs/raid56.c | 44 +++++++++--- fs/btrfs/scrub.c | 4 +- fs/btrfs/volumes.c | 170 ++++++++++++++++++++------------------------- fs/btrfs/volumes.h | 23 +++--- 4 files changed, 126 insertions(+), 115 deletions(-)
fs/btrfs/raid56.c +36 -8
@@ -912,7 +912,7 @@ static struct sector_ptr *sector_in_rbio(struct btrfs_raid_bio *rbio,static struct btrfs_raid_bio *alloc_rbio (struct btrfs_fs_info *fs_info,struct btrfs_io_context *bioc) { -const unsigned int real_stripes = bioc->num_stripes - bioc->num_tgtdevs; +const unsigned int real_stripes = bioc->num_stripes - bioc->replace_nr_stripes;const unsigned int stripe_npages =BTRFS_STRIPE_LEN >>PAGE_SHIFT ;const unsigned int num_pages = stripe_npages * real_stripes; const unsigned int stripe_nsectors =
@@ -1275,10 +1275,20 @@ static int rmw_assemble_write_bios(struct btrfs_raid_bio *rbio, goto error; } -if (likely (!rbio->bioc->num_tgtdevs)) +if (likely (!rbio->bioc->replace_nr_stripes))return 0 ; -/* Make a copy for the replace target device. */ +/* +* Make a copy for the replace target device. +* +* Thus the source stripe number (in replace_stripe_src[0]) +* should be valid. +* And just in case, make sure the second slot is uninitialized +* (-1), as there can only be one device being replace in RAID56. +*/ +ASSERT (rbio->bioc->replace_stripe_src[0 ] >=0 ); +ASSERT (rbio->bioc->replace_stripe_src[1 ] <0 ); +for (total_sector_nr =0 ; total_sector_nr < rbio->nr_sectors; total_sector_nr++) {struct sector_ptr *sector;
@@ -1286,7 +1296,12 @@ static int rmw_assemble_write_bios(struct btrfs_raid_bio *rbio, stripe = total_sector_nr / rbio->stripe_nsectors; sectornr = total_sector_nr % rbio->stripe_nsectors; -if (!rbio->bioc->tgtdev_map[stripe]) { +/* +* For RAID56, there is only one device can be replaced, +* and replace_stripe_src[0] indicates the stripe number +* we need to copy from. +*/ +if (stripe != rbio->bioc->replace_stripe_src[0 ]) {/* * We can skip the whole stripe completely, note * total_sector_nr will be increased by one anyway.
@@ -1309,7 +1324,7 @@ static int rmw_assemble_write_bios(struct btrfs_raid_bio *rbio, } ret =rbio_add_io_sector (rbio, bio_list, sector, - rbio->bioc->tgtdev_map[stripe], + rbio->real_stripes, sectornr,REQ_OP_WRITE );if (ret) goto error;
@@ -2518,7 +2533,12 @@ static int finish_parity_scrub(struct btrfs_raid_bio *rbio, int need_check)else BUG (); -if (bioc->num_tgtdevs && bioc->tgtdev_map[rbio->scrubp]) { +/* +* Replace is running and our P/Q stripe is being replace, then +* we need to duplicated the final write to reaplce target. +*/ +if (bioc->replace_nr_stripes && + bioc->replace_stripe_src[0 ] == rbio->scrubp) { is_replace =1 ;bitmap_copy (pbitmap, &rbio->dbitmap, rbio->stripe_nsectors); }
@@ -2620,13 +2640,21 @@ static int finish_parity_scrub(struct btrfs_raid_bio *rbio, int need_check)if (!is_replace) goto submit_write; +/* +* Replace is running and our parity stripe needs to be duplicated +* to target device. +* Checks we have valid source stripe number in the first slot, +* and the second slot is unused. +*/ +ASSERT (rbio->bioc->replace_stripe_src[0 ] >=0 ); +ASSERT (rbio->bioc->replace_stripe_src[1 ] <0 );for_each_set_bit (sectornr, pbitmap, rbio->stripe_nsectors) {struct sector_ptr *sector; sector =rbio_stripe_sector (rbio, rbio->scrubp, sectornr); ret =rbio_add_io_sector (rbio, &bio_list, sector, - bioc->tgtdev_map[rbio->scrubp], - sectornr,REQ_OP_WRITE ); + rbio->real_stripes, + sectornr,REQ_OP_WRITE );if (ret) goto cleanup; }
fs/btrfs/scrub.c +2 -2
@@ -1230,7 +1230,7 @@ static int scrub_handle_errored_block(struct scrub_block *sblock_to_check) sblock_other = sblocks_for_recheck[mirror_index]; }else {struct scrub_recover *r = sblock_bad->sectors[0 ]->recover; -int max_allowed = r->bioc->num_stripes - r->bioc->num_tgtdevs; +int max_allowed = r->bioc->num_stripes - r->bioc->replace_nr_stripes;if (mirror_index >= max_allowed)break ;
@@ -1540,7 +1540,7 @@ static int scrub_setup_recheck_block(struct scrub_block *original_sblock, bioc->map_type, bioc->raid_map, bioc->num_stripes - - bioc->num_tgtdevs, + bioc->replace_nr_stripes, mirror_index, &stripe_index, &stripe_offset);
fs/btrfs/volumes.c +77 -93
232 lines not on this page · raw message
fs/btrfs/volumes.h +11 -12
@@ -438,14 +438,13 @@ struct btrfs_io_context {/* * The following two members are for dev-replace case only. * -* @num_tgtdevs: Number of duplicated stripes which needs to be +* @replace_nr_stripes: Number of duplicated stripes which needs to be * written to replace target. * Should be <= 2 (2 for DUP, otherwise <= 1). -* @tgtdev_map: The array indicates where the duplicated stripes -* are from. The size is the number of original -* stripes (num_stripes - num_tgtdevs). +* @replace_stripe_src: The array indicates where the duplicated stripes +* are from. * -* The @tgtdev_map[] array is mostly for RAID56 cases. +* The @replace_stripe_src[] array is mostly for RAID56 cases. * As non-RAID56 stripes share the same contents for the mapped range, * thus no need to bother where the duplicated ones are from. *
@@ -459,14 +458,14 @@ struct btrfs_io_context {* stripes[2]: dev = devid 3, physical = Z * stripes[3]: dev = devid 0, physical = Y * -* num_tgtdevs = 1 -* tgtdev_map[0] = 0 <- Means stripes[0] is not involved in replace. -* tgtdev_map[1] = 3 <- Means stripes[1] is involved in replace, -* and it's duplicated to stripes[3]. -* tgtdev_map[2] = 0 <- Means stripes[2] is not involved in replace. +* replace_nr_stripes = 1 +* tgtdev_map[0] = 1 <- Means stripes[1] is involved in replace. +* tgtdev_map[1] = -1 <- Means the second slot is not used. +* (Only DUP can use this slot, but they +* don't really care) */ -u16 num_tgtdevs; -u16 *tgtdev_map; +u16 replace_nr_stripes; +s16 replace_stripe_src[2 ];/* * logical block numbers for the start of each stripe * The last one or two are p/q. These are sorted,
Signature · 1 line
2.39.1
Re: [PATCH 3/3] btrfs: use a more space efficient way to represent the source of duplicated stripes
fs/btrfs/raid56.c static int rmw_assemble_write_bios(struct btrfs_raid_bio *rbio, · line 1300
Quoted 4 lines · Qu Wenruo
fs/btrfs/raid56.c +2
-1-if (!rbio->bioc->tgtdev_map[stripe]) { +/* +* For RAID56, there is only one device can be replaced,
Nit: one device _that_ can be
fs/btrfs/raid56.c static int finish_parity_scrub(struct btrfs_raid_bio *rbio, int need_check) · line 2537
Quoted 13 lines
fs/btrfs/raid56.c +3
-2ret =rbio_add_io_sector (rbio, bio_list, sector, - rbio->bioc->tgtdev_map[stripe], + rbio->real_stripes, sectornr,REQ_OP_WRITE );if (ret) goto error;@@ -2518,7 +2533,12 @@ static int finish_parity_scrub(struct btrfs_raid_bio *rbio, int need_check)else BUG (); -if (bioc->num_tgtdevs && bioc->tgtdev_map[rbio->scrubp]) { +/* +* Replace is running and our P/Q stripe is being replace, then
being replaced.
fs/btrfs/raid56.c static int finish_parity_scrub(struct btrfs_raid_bio *rbio, int need_check) · line 2538
Quoted 1 line
fs/btrfs/raid56.c +1
-0+* we need to duplicated the final write to reaplce target.
s/reaplce/replace/
fs/btrfs/raid56.c static int finish_parity_scrub(struct btrfs_raid_bio *rbio, int need_check) · line 2645
Quoted 7 lines
fs/btrfs/raid56.c +3
-0@@ -2620,13 +2640,21 @@ static int finish_parity_scrub(struct btrfs_raid_bio *rbio, int need_check)if (!is_replace) goto submit_write; +/* +* Replace is running and our parity stripe needs to be duplicated +* to target device.
to _the_ target device.
Quoted 13 lines
fs/btrfs/raid56.c +7
-1+* Checks we have valid source stripe number in the first slot, +* and the second slot is unused. +*/ +ASSERT (rbio->bioc->replace_stripe_src[0 ] >=0 );@@ -6183,93 +6181,77 @@ static void handle_ops_on_dev_replace(enum btrfs_map_op op, int *num_stripes_ret, int *max_errors_ret) { u64 srcdev_devid = dev_replace->srcdev->devid; -int tgtdev_indexes =0 ; +/* +* At this stage, num_stripes is still the real number of stripes, +* excluding the duplicated stripes.
fs/btrfs/raid56.c static void handle_ops_on_dev_replace(enum btrfs_map_op op, · line 6194
Quoted 10 lines
fs/btrfs/raid56.c +4
-2+*/ int num_stripes = *num_stripes_ret; +int nr_extra_stripes =0 ;int max_errors = *max_errors_ret;int i; -if (op ==BTRFS_MAP_WRITE ) { -int index_where_to_add; +/* +* A block group which have "to_copy" set will eventually
s/have/has/
fs/btrfs/volumes.c static void handle_ops_on_dev_replace(enum btrfs_map_op op, · line 6195
Quoted 1 line
fs/btrfs/volumes.c +1
-0+* copied by dev-replace process. We can avoid cloning IO here.
be copied by the dev-replace process.
fs/btrfs/volumes.c static void handle_ops_on_dev_replace(enum btrfs_map_op op, · line 6201
Quoted 6 lines
fs/btrfs/volumes.c +6
-0+*/ +if (is_block_group_to_copy (dev_replace->srcdev->fs_info, logical)) +return ; + +/* +* duplicate the write operations while the dev replace
Duplicate
Quoted 1 line
fs/btrfs/volumes.c +1
-0+* procedure is running. Since the copying of the old disk to