From: Guoqing Jiang Subject: Re: [PATCH V2 3/3] improve raid10 discard request Date: Tue, 28 Jul 2020 10:38:16 +0200 Message-ID: <81fdb14b-e292-8dfc-5228-c6bfaa800dfe@cloud.ionos.com> References: <1595920703-6125-1-git-send-email-xni@redhat.com> <1595920703-6125-4-git-send-email-xni@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Return-path: In-Reply-To: <1595920703-6125-4-git-send-email-xni@redhat.com> Content-Language: en-US Sender: linux-raid-owner@vger.kernel.org To: Xiao Ni , song@kernel.org, linux-raid@vger.kernel.org Cc: colyli@suse.de, ncroxon@redhat.com, heinzm@redhat.com List-Id: linux-raid.ids On 7/28/20 9:18 AM, Xiao Ni wrote: > Now the discard request is split by chunk size. So it takes a long time > to finish mkfs on disks which support discard function. This patch improve > handling raid10 discard request. It uses the similar way with patch > 29efc390b (md/md0: optimize raid0 discard handling). > > But it's a little complex than raid0. Because raid10 has different layout. > If raid10 is offset layout and the discard request is smaller than stripe > size. There are some holes when we submit discard bio to underlayer disks. > > For example: five disks (disk1 - disk5) > D01 D02 D03 D04 D05 > D05 D01 D02 D03 D04 > D06 D07 D08 D09 D10 > D10 D06 D07 D08 D09 > The discard bio just wants to discard from D03 to D10. For disk3, there is > a hole between D03 and D08. For disk4, there is a hole between D04 and D09. > D03 is a chunk, raid10_write_request can handle one chunk perfectly. So > the part that is not aligned with stripe size is still handled by > raid10_write_request. One question, is far layout handled by raid10_write_request or the discard request? > > If reshape is running when discard bio comes and the discard bio spans the > reshape position, raid10_write_request is responsible to handle this > discard bio. > > I did a test with this patch set. > Without patch: > time mkfs.xfs /dev/md0 > real4m39.775s > user0m0.000s > sys0m0.298s > > With patch: > time mkfs.xfs /dev/md0 > real0m0.105s > user0m0.000s > sys0m0.007s > > nvme3n1 259:1 0 477G 0 disk > └─nvme3n1p1 259:10 0 50G 0 part > nvme4n1 259:2 0 477G 0 disk > └─nvme4n1p1 259:11 0 50G 0 part > nvme5n1 259:6 0 477G 0 disk > └─nvme5n1p1 259:12 0 50G 0 part > nvme2n1 259:9 0 477G 0 disk > └─nvme2n1p1 259:15 0 50G 0 part > nvme0n1 259:13 0 477G 0 disk > └─nvme0n1p1 259:14 0 50G 0 part > > v1: > Coly helps to review these patches and give some suggestions: > One bug is found. If discard bio is across one stripe but bio size is bigger > than one stripe size. After spliting, the bio will be NULL. In this version, > it checks whether discard bio size is bigger than 2*stripe_size. > In raid10_end_discard_request, it's better to check R10BIO_Uptodate is set > or not. It can avoid write memory to improve performance. > Add more comments for calculating addresses. > > v2: > Fix error by checkpatch.pl > Fix one bug for offset layout. v1 calculates wrongly split size > Add more comments to explain how the discard range of each component disk > is decided. The above change log are usually put under "---". > Reviewed-by: Coly Li > Signed-off-by: Xiao Ni > --- [...] > + > +/* There are some limitations to handle discard bio > + * 1st, the discard size is bigger than stripe_size*2. > + * 2st, if the discard bio spans reshape progress, we use the old way to > + * handle discard bio > + */ > +static bool raid10_handle_discard(struct mddev *mddev, struct bio *bio) > +{ [...] > + /* For far offset layout, if bio is not aligned with stripe size, it splits > + * the part that is not aligned with strip size. > + */ > + if (geo.far_offset && (bio_start & stripe_mask)) { > + sector_t split_size; > + split_size = round_up(bio_start, stripe_size) - bio_start; > + bio = raid10_split_bio(conf, bio, split_size, false); > + } > + if (geo.far_offset && (bio_end & stripe_mask)) { > + sector_t split_size; > + split_size = bio_sectors(bio) - (bio_end & stripe_mask); > + bio = raid10_split_bio(conf, bio, split_size, true); > + } So far layout is handled here. I think the hole issue is existed for far layout, Just FYI. > + > + r10_bio = mempool_alloc(&conf->r10bio_pool, GFP_NOIO); > + r10_bio->mddev = mddev; > + r10_bio->state = 0; > + memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * conf->geo.raid_disks); > + > + wait_blocked_dev(mddev, geo.raid_disks); > + > + r10_bio->master_bio = bio; > + > + bio_start = bio->bi_iter.bi_sector; > + bio_end = bio_end_sector(bio); > + > + /* raid10 uses chunk as the unit to store data. It's similar like raid0. > + * One stripe contains the chunks from all member disk (one chunk from > + * one disk at the same HAB address). For layout detail, see 'man md 4' > + */ s/HAB/HBA/ Thanks, Guoqing