Skip to contentSkip to conversation

btrfs: add subpage support for RAID56

PATCH In review Qu Wenruo 4 participants 34 messages Apr 12, 2022 – Apr 21, 2022 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

v2 cover btrfs: add subpage support for RAID56

The branch can be fetched from github, based on latest misc-next branch (with bio and memory allocation refactors):
https://github.com/adam900710/linux/tree/subpage_raid56

[CHANGELOG]
v2:
- Rebased to latest misc-next
There are several conflicts caused by bio interface change and page allocation update.

- A new patch to reduce the width of @stripe_len to u32 Currently @stripe_len is fixed to 64K, and even in the future we choose to enlarge the value, I see no reason to go beyond 4G for stripe length.

Thus change it u32 to avoid some u64-divided-by-u32 situations.

This will reduce memory usage for map_lookup (which has a lifespan as long as the mounted fs) and btrfs_io_geometry (which only has a very short lifespan, mostly bounded to bio).

Furthermore, add some extra alignment check and use right bit shift to replace involved division to avoid possible problems on 32bit systems.

- Pack sector_ptr::pgoff and sector_ptr::uptodate into one u32 This will reduce memory usage and reduce unaligned memory access

Please note that, even with it packed, we still have a 4 bytes padding (it's u64 + u32, thus not perfectly aligned).
Without packed attribute, it will cost more memory usage anyway.

- Call kunmap_local() using address with pgoff
As it can handle it without problem, no need to bother extra search just for pgoff.

- Use "= { 0 }" for structure initialization

- Reduce comment updates to minimal
If one comment line is not really touched, then don't touch it just to fix some bad styles.

[DESIGN]
To make RAID56 code to support subpage, we need to make every sector of a RAID56 full stripe (including P/Q) to be addressable.

Previously we use page pointer directly for things like stripe_pages:

Full stripe is 64K * 3, 2 data stripes, one P stripe (RAID5)

stripe_pages: | 0 | 1 | 2 | .. | 46 | 47 |

Those 48 pages all points to a page we allocated.

The new subpage support will introduce a sector layer, based on the old stripe_pages[] array:

The same 64K * 3, RAID5 layout, but 64K page size and 4K sector size:

stripe_sectors: | 0 | 1 | .. |15 |16 |  ...  |31 |32 | ..    |47 |
stripe_pages:   |      Page 0    |     Page 1    |    Page 2     |

Each stripe_ptr of stripe_sectors[] will include:

- One page pointer
Points back the page inside stripe_pages[].

- One pgoff member
To indicate the offset inside the page

- One uptodate member
To indicate if the sector is uptodate, replacing the old PageUptodate flag.
As introducing btrfs_subpage structure to stripe_pages[] looks a little overkilled, as we only care Uptodate flag.

The same applies to bio_sectors[] array, which is going to completely replace the old bio_pages[] array.

[SIDE EFFECT]
Despite the obvious new ability for subpage to utilize btrfs RAID56 profiles, it will cause more memory usage for real btrfs_raid_bio structure.

We allocate extra memory based on the stripe size and number of stripes, and update the pointer arrays to utilize the extra memory.

To compare, we use a pretty standard setup, 3 disks raid5, 4K page size on x86_64:

Before: 1176
After: 2320 (+97.8%)

The reason for such a big bump is:

- Extra size for sector_ptr.
Instead of just a page pointer, now it's twice the size of a pointer (a page pointer + 2 * unsigned int)

This means although we replace bio_pages[] with bio_sectors[], we are still enlarging the size.

- A completely new array for stripe_sectors[]
And the array itself is also twice the size of the old stripe_pages[].

- Extra padding for sector_ptr
Since we don't have packed attribute anymore, the real size of a sector_ptr is 16 bytes, not 12 bytes.

There is some attempt to reduce the size of btrfs_raid_bio itself, but the big impact still comes from the new sector_ptr arrays.

Without exotic macros or packed attribute, I don't have any better ideas on reducing the real size of btrfs_raid_bio.

[TESTS]
Now due to recent new error path exposed in generic/475, only btrfs groups are tested. Or it will always hang at generic/475 due to unrelated bugs.

Both x86_64 and aarch64 (64K page size) pass the full btrfs test cases without new regression.

[PATCHSET LAYOUT]
The patchset layout puts several things into consideration:

- Every patch can be tested independently on x86_64 No more tons of unused helpers then a big switch. Every change can be verified on x86_64.

- More temporary sanity checks than previous code
For example, when rbio_add_io_page() is converted to be subpage compatible, extra ASSERT() is added to ensure no subpage range can even be added.

Such temporary checks are removed in the last enablement patch. This is to make testing on x86_64 more comprehensive.

- Mostly small change in each patch
The only exception is the conversion for rbio_add_io_page(). But the most change in that patch comes from variable renaming. The overall line changed in each patch should still be small enough for review.

Qu Wenruo (17):
btrfs: reduce width for stripe_len from u64 to u32
btrfs: open-code rbio_nr_pages()
btrfs: make btrfs_raid_bio more compact
btrfs: introduce new cached members for btrfs_raid_bio
btrfs: introduce btrfs_raid_bio::stripe_sectors
btrfs: introduce btrfs_raid_bio::bio_sectors
btrfs: make rbio_add_io_page() subpage compatible
btrfs: make finish_parity_scrub() subpage compatible
btrfs: make __raid_recover_endio_io() subpage compatibable
btrfs: make finish_rmw() subpage compatible
btrfs: open-code rbio_stripe_page_index()
btrfs: make raid56_add_scrub_pages() subpage compatible
btrfs: remove btrfs_raid_bio::bio_pages array
btrfs: make set_bio_pages_uptodate() subpage compatible
btrfs: make steal_rbio() subpage compatible
btrfs: make alloc_rbio_essential_pages() subpage compatible
btrfs: enable subpage support for RAID56

 fs/btrfs/disk-io.c |   8 -
 fs/btrfs/raid56.c  | 749 +++++++++++++++++++++++++++------------------
 fs/btrfs/raid56.h  |   8 +-
 fs/btrfs/scrub.c   |   6 +-
 fs/btrfs/volumes.c |  27 +-
 fs/btrfs/volumes.h |   8 +-
 6 files changed, 479 insertions(+), 327 deletions(-)
Signature · 1 line
2.35.1
author
#1 lore raw

PATCH v2 1/17 btrfs: reduce width for stripe_len from u64 to u32

Currently btrfs uses fixed stripe length (64K), thus u32 is wide enough for the usage.

Furthermore, even in the future we choose to enlarge stripe length to larger values, I don't believe we would want stripe as large as 4G or larger.

So this patch will reduce the width for all in-memory structures and parameters, this involves:

- RAID56 related function argument lists
This allows us to do direct division related to stripe_len. Although we will use bits shift to replace the division anyway.

- btrfs_io_geometry structure
This involves one change to simplify the calculation of both @stripe_nr and @stripe_offset, using div64_u64_rem().
And add extra sanity check to make sure @stripe_offset is always small enough for u32.

This saves 8 bytes for the structure.

- map_lookup structure
This convert @stripe_len to u32, which saves 8 bytes. (saved 4 bytes, and removed a 4-bytes hole)

---
 fs/btrfs/raid56.c  | 15 ++++++++-------
 fs/btrfs/raid56.h  |  6 +++---
 fs/btrfs/volumes.c | 20 +++++++-------------
 fs/btrfs/volumes.h |  8 ++++----
 4 files changed, 22 insertions(+), 27 deletions(-)
fs/btrfs/raid56.c +8 -7
index 79438cdd604e..daffa3076325 100644
@@ -949,9 +949,10 @@ static struct page *page_in_rbio(struct btrfs_raid_bio *rbio,
  * number of pages we need for the entire stripe across all the
  * drives
  */
-static unsigned long rbio_nr_pages(unsigned long stripe_len, int nr_stripes)
+static unsigned long rbio_nr_pages(u32 stripe_len, int nr_stripes)
 {
-	return DIV_ROUND_UP(stripe_len, PAGE_SIZE) * nr_stripes;
+	ASSERT(IS_ALIGNED(stripe_len, PAGE_SIZE));
+	return (stripe_len >> PAGE_SHIFT) * nr_stripes;
 }
 
 /*
@@ -960,13 +961,13 @@ static unsigned long rbio_nr_pages(unsigned long stripe_len, int nr_stripes)
  */
 static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 					 struct btrfs_io_context *bioc,
-					 u64 stripe_len)
+					 u32 stripe_len)
 {
 	struct btrfs_raid_bio *rbio;
 	int nr_data = 0;
 	int real_stripes = bioc->num_stripes - bioc->num_tgtdevs;
 	int num_pages = rbio_nr_pages(stripe_len, real_stripes);
-	int stripe_npages = DIV_ROUND_UP(stripe_len, PAGE_SIZE);
+	int stripe_npages = stripe_len >> PAGE_SHIFT;
 	void *p;
 
 	rbio = kzalloc(sizeof(*rbio) +
@@ -1692,7 +1693,7 @@ static void btrfs_raid_unplug(struct blk_plug_cb *cb, bool from_schedule)
  * our main entry point for writes from the rest of the FS.
  */
 int raid56_parity_write(struct bio *bio, struct btrfs_io_context *bioc,
-			u64 stripe_len)
+			u32 stripe_len)
 {
 	struct btrfs_fs_info *fs_info = bioc->fs_info;
 	struct btrfs_raid_bio *rbio;
@@ -2089,7 +2090,7 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)
  * of the drive.
  */
 int raid56_parity_recover(struct bio *bio, struct btrfs_io_context *bioc,
-			  u64 stripe_len, int mirror_num, int generic_io)
+			  u32 stripe_len, int mirror_num, int generic_io)
 {
 	struct btrfs_fs_info *fs_info = bioc->fs_info;
 	struct btrfs_raid_bio *rbio;
@@ -2195,7 +2196,7 @@ static void read_rebuild_work(struct btrfs_work *work)
 
 struct btrfs_raid_bio *raid56_parity_alloc_scrub_rbio(struct bio *bio,
 				struct btrfs_io_context *bioc,
-				u64 stripe_len, struct btrfs_device *scrub_dev,
+				u32 stripe_len, struct btrfs_device *scrub_dev,
 				unsigned long *dbitmap, int stripe_nsectors)
 {
 	struct btrfs_fs_info *fs_info = bioc->fs_info;
fs/btrfs/raid56.h +3 -3
index 72c00fc284b5..fb35ae157b02 100644
@@ -31,15 +31,15 @@ struct btrfs_raid_bio;
 struct btrfs_device;
 
 int raid56_parity_recover(struct bio *bio, struct btrfs_io_context *bioc,
-			  u64 stripe_len, int mirror_num, int generic_io);
+			  u32 stripe_len, int mirror_num, int generic_io);
 int raid56_parity_write(struct bio *bio, struct btrfs_io_context *bioc,
-			u64 stripe_len);
+			u32 stripe_len);
 
 void raid56_add_scrub_pages(struct btrfs_raid_bio *rbio, struct page *page,
 			    u64 logical);
 
 struct btrfs_raid_bio *raid56_parity_alloc_scrub_rbio(struct bio *bio,
-				struct btrfs_io_context *bioc, u64 stripe_len,
+				struct btrfs_io_context *bioc, u32 stripe_len,
 				struct btrfs_device *scrub_dev,
 				unsigned long *dbitmap, int stripe_nsectors);
 void raid56_parity_submit_scrub_rbio(struct btrfs_raid_bio *rbio);
fs/btrfs/volumes.c +7 -13
index 2368a2ffbee7..93072a090fdd 100644
@@ -6313,7 +6313,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
 	u64 offset;
 	u64 stripe_offset;
 	u64 stripe_nr;
-	u64 stripe_len;
+	u32 stripe_len;
 	u64 raid56_full_stripe_start = (u64)-1;
 	int data_stripes;
 
@@ -6324,19 +6324,13 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
 	offset = logical - em->start;
 	/* Len of a stripe in a chunk */
 	stripe_len = map->stripe_len;
-	/* Stripe where this block falls in */
-	stripe_nr = div64_u64(offset, stripe_len);
-	/* Offset of stripe in the chunk */
-	stripe_offset = stripe_nr * stripe_len;
-	if (offset < stripe_offset) {
-		btrfs_crit(fs_info,
-"stripe math has gone wrong, stripe_offset=%llu offset=%llu start=%llu logical=%llu stripe_len=%llu",
-			stripe_offset, offset, em->start, logical, stripe_len);
-		return -EINVAL;
-	}
+	/*
+	 * Stripe_nr is where this block falls in
+	 * stripe_offset is the offset of this block in its stripe.
+	 */
+	stripe_nr = div64_u64_rem(offset, stripe_len, &stripe_offset);
+	ASSERT(stripe_offset < U32_MAX);
 
-	/* stripe_offset is the offset of this block in its stripe */
-	stripe_offset = offset - stripe_offset;
 	data_stripes = nr_data_stripes(map);
 
 	/* Only stripe based profiles needs to check against stripe length. */
fs/btrfs/volumes.h +4 -4
index bd297f23d19e..0332a0a25483 100644
@@ -23,11 +23,11 @@ struct btrfs_io_geometry {
 	/* offset of logical address in chunk */
 	u64 offset;
 	/* length of single IO stripe */
-	u64 stripe_len;
+	u32 stripe_len;
+	/* offset of address in stripe */
+	u32 stripe_offset;
 	/* number of stripe where address falls */
 	u64 stripe_nr;
-	/* offset of address in stripe */
-	u64 stripe_offset;
 	/* offset of raid56 stripe into the chunk */
 	u64 raid56_stripe_offset;
 };
@@ -427,7 +427,7 @@ struct map_lookup {
 	u64 type;
 	int io_align;
 	int io_width;
-	u64 stripe_len;
+	u32 stripe_len;
 	int num_stripes;
 	int sub_stripes;
 	int verified_stripes; /* For mount time dev extent verification */
Signature · 1 line
2.35.1
author
#2 lore raw

PATCH v2 2/17 btrfs: open-code rbio_nr_pages()

In reply to Qu Wenruo

The function rbio_nr_pages() is only called once inside alloc_rbio(), there is no reason to make it dedicated helper.

Furthermore, the return type doesn't match, the function return "unsigned long" which may not be necessary, while the only caller only uses "int".

Since we're doing cleaning up here, also fix the type to "const unsigned int" for all involved local variables.

This change will slightly modify the timing of the ASSERT() on @stripe_len though.

---
 fs/btrfs/raid56.c | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)
fs/btrfs/raid56.c +5 -13
index daffa3076325..a16297b04db6 100644
@@ -945,16 +945,6 @@ static struct page *page_in_rbio(struct btrfs_raid_bio *rbio,
 	return rbio->stripe_pages[chunk_page];
 }
 
-/*
- * number of pages we need for the entire stripe across all the
- * drives
- */
-static unsigned long rbio_nr_pages(u32 stripe_len, int nr_stripes)
-{
-	ASSERT(IS_ALIGNED(stripe_len, PAGE_SIZE));
-	return (stripe_len >> PAGE_SHIFT) * nr_stripes;
-}
-
 /*
  * allocation and initial setup for the btrfs_raid_bio.  Not
  * this does not allocate any pages for rbio->pages.
@@ -963,13 +953,15 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 					 struct btrfs_io_context *bioc,
 					 u32 stripe_len)
 {
+	const unsigned int real_stripes = bioc->num_stripes - bioc->num_tgtdevs;
+	const unsigned int stripe_npages = stripe_len >> PAGE_SHIFT;
+	const unsigned int num_pages = stripe_npages * real_stripes;
 	struct btrfs_raid_bio *rbio;
 	int nr_data = 0;
-	int real_stripes = bioc->num_stripes - bioc->num_tgtdevs;
-	int num_pages = rbio_nr_pages(stripe_len, real_stripes);
-	int stripe_npages = stripe_len >> PAGE_SHIFT;
 	void *p;
 
+	ASSERT(IS_ALIGNED(stripe_len, PAGE_SIZE));
+
 	rbio = kzalloc(sizeof(*rbio) +
 		       sizeof(*rbio->stripe_pages) * num_pages +
 		       sizeof(*rbio->bio_pages) * num_pages +
Signature · 1 line
2.35.1
author
#3 lore raw

PATCH v2 3/17 btrfs: make btrfs_raid_bio more compact

In reply to Qu Wenruo

There are a lot of members using much larger type in btrfs_raid_bio.

Like nr_pages which represents the total number of a full stripe.

Instead of int (which is at least 32bits), u16 is already enough (max stripe length will be 256MiB, already beyond current raid56 device number limit).

So this patch will reduce the width of the following members:

- stripe_len to u32
- nr_pages to u16
- nr_data to u8
- real_stripes to u8
- scrubp to u8
- faila/b to s8
As -1 is used to indicate no corruption

This will slightly reduce the size of btrfs_raid_bio from 272 bytes to 256 bytes, reducing 16 bytes usage.

But please note that, when using btrfs_raid_bio, we allocate extra space for it to cover various pointer array, so the reduce memory is not really a big saving overall.

As we're here modifying the comments already, update existing comments to current code standard.

---
 fs/btrfs/raid56.c | 42 ++++++++++++++++++++++--------------------
 1 file changed, 22 insertions(+), 20 deletions(-)
fs/btrfs/raid56.c +22 -20
index a16297b04db6..8cf0f368729a 100644
@@ -101,15 +101,6 @@ struct btrfs_raid_bio {
 	 */
 	unsigned long flags;
 
-	/* size of each individual stripe on disk */
-	int stripe_len;
-
-	/* number of data stripes (no p/q) */
-	int nr_data;
-
-	int real_stripes;
-
-	int stripe_npages;
 	/*
 	 * set if we're doing a parity rebuild
 	 * for a read from higher up, which is handled
@@ -118,21 +109,32 @@ struct btrfs_raid_bio {
 	 */
 	enum btrfs_rbio_ops operation;
 
-	/* first bad stripe */
-	int faila;
+	/* Size of each individual stripe on disk */
+	u32 stripe_len;
 
-	/* second bad stripe (for raid6 use) */
-	int failb;
+	/* How many pages there are for the full stripe including P/Q */
+	u16 nr_pages;
 
-	int scrubp;
-	/*
-	 * number of pages needed to represent the full
-	 * stripe
-	 */
-	int nr_pages;
+	/* Number of data stripes (no p/q) */
+	u8 nr_data;
+
+	/* Numer of all stripes (including P/Q) */
+	u8 real_stripes;
+
+	/* How many pages there are for each stripe */
+	u8 stripe_npages;
+
+	/* First bad stripe, -1 means no corruption */
+	s8 faila;
+
+	/* Second bad stripe (for raid6 use) */
+	s8 failb;
+
+	/* Stripe number that we're scrubbing  */
+	u8 scrubp;
 
 	/*
-	 * size of all the bios in the bio_list.  This
+	 * Size of all the bios in the bio_list.  This
 	 * helps us decide if the rbio maps to a full
 	 * stripe or not
 	 */
Signature · 1 line
2.35.1
author
#4 lore raw

PATCH v2 4/17 btrfs: introduce new cached members for btrfs_raid_bio

In reply to Qu Wenruo

The new members are all related to number of sectors, but the existing number of pages members are kept as is:

- nr_sectors
Total sectors of the full stripe including P/Q.

- stripe_nsectors
The sectors of a single stripe.

---
 fs/btrfs/raid56.c | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
fs/btrfs/raid56.c +18 -5
index 8cf0f368729a..92d464d4456a 100644
@@ -115,6 +115,9 @@ struct btrfs_raid_bio {
 	/* How many pages there are for the full stripe including P/Q */
 	u16 nr_pages;
 
+	/* How many sectors there are for the full stripe including P/Q */
+	u16 nr_sectors;
+
 	/* Number of data stripes (no p/q) */
 	u8 nr_data;
 
@@ -124,6 +127,9 @@ struct btrfs_raid_bio {
 	/* How many pages there are for each stripe */
 	u8 stripe_npages;
 
+	/* How many sectors there are for each stripe */
+	u8 stripe_nsectors;
+
 	/* First bad stripe, -1 means no corruption */
 	s8 faila;
 
@@ -172,7 +178,7 @@ struct btrfs_raid_bio {
 	/* allocated with real_stripes-many pointers for finish_*() calls */
 	void **finish_pointers;
 
-	/* allocated with stripe_npages-many bits for finish_*() calls */
+	/* Allocated with stripe_nsectors-many bits for finish_*() calls */
 	unsigned long *finish_pbitmap;
 };
 
@@ -958,19 +964,24 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	const unsigned int real_stripes = bioc->num_stripes - bioc->num_tgtdevs;
 	const unsigned int stripe_npages = stripe_len >> PAGE_SHIFT;
 	const unsigned int num_pages = stripe_npages * real_stripes;
+	const unsigned int stripe_nsectors = stripe_len >>
+					     fs_info->sectorsize_bits;
+	const unsigned int num_sectors = stripe_nsectors * real_stripes;
 	struct btrfs_raid_bio *rbio;
 	int nr_data = 0;
 	void *p;
 
 	ASSERT(IS_ALIGNED(stripe_len, PAGE_SIZE));
+	/* PAGE_SIZE must also be aligned to sectorsize for subpage support */
+	ASSERT(IS_ALIGNED(PAGE_SIZE, fs_info->sectorsize));
 
 	rbio = kzalloc(sizeof(*rbio) +
 		       sizeof(*rbio->stripe_pages) * num_pages +
 		       sizeof(*rbio->bio_pages) * num_pages +
 		       sizeof(*rbio->finish_pointers) * real_stripes +
-		       sizeof(*rbio->dbitmap) * BITS_TO_LONGS(stripe_npages) +
+		       sizeof(*rbio->dbitmap) * BITS_TO_LONGS(stripe_nsectors) +
 		       sizeof(*rbio->finish_pbitmap) *
-				BITS_TO_LONGS(stripe_npages),
+				BITS_TO_LONGS(stripe_nsectors),
 		       GFP_NOFS);
 	if (!rbio)
 		return ERR_PTR(-ENOMEM);
@@ -983,8 +994,10 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	rbio->bioc = bioc;
 	rbio->stripe_len = stripe_len;
 	rbio->nr_pages = num_pages;
+	rbio->nr_sectors = num_sectors;
 	rbio->real_stripes = real_stripes;
 	rbio->stripe_npages = stripe_npages;
+	rbio->stripe_nsectors = stripe_nsectors;
 	rbio->faila = -1;
 	rbio->failb = -1;
 	refcount_set(&rbio->refs, 1);
@@ -1003,8 +1016,8 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	CONSUME_ALLOC(rbio->stripe_pages, num_pages);
 	CONSUME_ALLOC(rbio->bio_pages, num_pages);
 	CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
-	CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_npages));
-	CONSUME_ALLOC(rbio->finish_pbitmap, BITS_TO_LONGS(stripe_npages));
+	CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_nsectors));
+	CONSUME_ALLOC(rbio->finish_pbitmap, BITS_TO_LONGS(stripe_nsectors));
 #undef  CONSUME_ALLOC
 
 	if (bioc->map_type & BTRFS_BLOCK_GROUP_RAID5)
Signature · 1 line
2.35.1
#5 lore raw

Re: [PATCH v2 04/17] btrfs: introduce new cached members for btrfs_raid_bio

What is cached about these members?

#6 lore raw

Re: [PATCH v2 04/17] btrfs: introduce new cached members for btrfs_raid_bio

Quoted 1 line · Christoph Hellwig

What is cached about these members?

It's just like btrfs_fs_info::sectorsize, gives us a quick way to use, without calculating the same value again and again.

BTW, are you suggesting to calculate everything when needed?

I'm fine either way.

Thanks,
Qu

#7 lore raw

Re: [PATCH v2 04/17] btrfs: introduce new cached members for btrfs_raid_bio

Quoted 11 lines · Qu Wenruo
Quoted 1 line · Christoph Hellwig

What is cached about these members?

It's just like btrfs_fs_info::sectorsize, gives us a quick way to use, without calculating the same value again and again.

BTW, are you suggesting to calculate everything when needed?

I'm fine either way.

No. I was just confused by the term cached.

author
#8 lore raw

PATCH v2 5/17 btrfs: introduce btrfs_raid_bio::stripe_sectors

In reply to Qu Wenruo

The new member is an array of sector_ptr pointers, they will represent all sectors inside a full stripe (including P/Q).

They co-operate with btrfs_raid_bio::stripe_pages:

stripe_pages:   | Page 0, range [0, 64K)   | Page 1 ...
stripe_sectors: |  |  | ...             |  |
                  |  |                    \- sector 15, page 0, pgoff=60K
                  |  \- sector 1, page 0, pgoff=4K
                  \---- sector 0, page 0, pfoff=0

With such structure, we can represent subpage sectors without using extra pages.

Here we introduce a new helper, index_stripe_sectors(), to update stripe_sectors[] to point to correct page and pgoff.

So every time rbio::stripe_pages[] pointer gets updated, the new helper should be called.

The following functions have to call the new helper:

- steal_rbio()
- alloc_rbio_pages()
- alloc_rbio_parity_pages()
- alloc_rbio_essential_pages()

---
 fs/btrfs/raid56.c | 65 +++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 60 insertions(+), 5 deletions(-)
fs/btrfs/raid56.c +60 -5
index 92d464d4456a..0c7b6a06d5ce 100644
@@ -52,6 +52,16 @@ struct btrfs_stripe_hash_table {
 	struct btrfs_stripe_hash table[];
 };
 
+/*
+ * A bvec like structure to present a sector inside a page.
+ *
+ * Unlike bvec we don't need bvlen, as it's fixed to sectorsize.
+ */
+struct sector_ptr {
+	struct page *page;
+	unsigned int pgoff;
+};
+
 enum btrfs_rbio_ops {
 	BTRFS_RBIO_WRITE,
 	BTRFS_RBIO_READ_REBUILD,
@@ -171,8 +181,12 @@ struct btrfs_raid_bio {
 	struct page **bio_pages;
 
 	/*
-	 * bitmap to record which horizontal stripe has data
+	 * For subpage support, we need to map each sector to above
+	 * stripe_pages.
 	 */
+	struct sector_ptr *stripe_sectors;
+
+	/* Bitmap to record which horizontal stripe has data */
 	unsigned long *dbitmap;
 
 	/* allocated with real_stripes-many pointers for finish_*() calls */
@@ -291,6 +305,26 @@ static int rbio_bucket(struct btrfs_raid_bio *rbio)
 	return hash_64(num >> 16, BTRFS_STRIPE_HASH_TABLE_BITS);
 }
 
+/*
+ * Update the stripe_sectors[] array to use correct page and pgoff
+ *
+ * Should be called every time any page pointer in stripes_pages[] got modified.
+ */
+static void index_stripe_sectors(struct btrfs_raid_bio *rbio)
+{
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
+	u32 offset;
+	int i;
+
+	for (i = 0, offset = 0; i < rbio->nr_sectors; i++, offset += sectorsize) {
+		int page_index = offset >> PAGE_SHIFT;
+
+		ASSERT(page_index < rbio->nr_pages);
+		rbio->stripe_sectors[i].page = rbio->stripe_pages[page_index];
+		rbio->stripe_sectors[i].pgoff = offset_in_page(offset);
+	}
+}
+
 /*
  * stealing an rbio means taking all the uptodate pages from the stripe
  * array in the source rbio and putting them into the destination rbio
@@ -317,6 +351,8 @@ static void steal_rbio(struct btrfs_raid_bio *src, struct btrfs_raid_bio *dest)
 		dest->stripe_pages[i] = s;
 		src->stripe_pages[i] = NULL;
 	}
+	index_stripe_sectors(dest);
+	index_stripe_sectors(src);
 }
 
 /*
@@ -978,6 +1014,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	rbio = kzalloc(sizeof(*rbio) +
 		       sizeof(*rbio->stripe_pages) * num_pages +
 		       sizeof(*rbio->bio_pages) * num_pages +
+		       sizeof(*rbio->stripe_sectors) * num_sectors +
 		       sizeof(*rbio->finish_pointers) * real_stripes +
 		       sizeof(*rbio->dbitmap) * BITS_TO_LONGS(stripe_nsectors) +
 		       sizeof(*rbio->finish_pbitmap) *
@@ -1015,6 +1052,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	} while (0)
 	CONSUME_ALLOC(rbio->stripe_pages, num_pages);
 	CONSUME_ALLOC(rbio->bio_pages, num_pages);
+	CONSUME_ALLOC(rbio->stripe_sectors, num_sectors);
 	CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
 	CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_nsectors));
 	CONSUME_ALLOC(rbio->finish_pbitmap, BITS_TO_LONGS(stripe_nsectors));
@@ -1031,19 +1069,35 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	return rbio;
 }
 
-/* allocate pages for all the stripes in the bio, including parity */
+/*
+ * Allocate pages for all the stripes in the bio including parity, and map all
+ * sectors to their corresponding pages
+ */
 static int alloc_rbio_pages(struct btrfs_raid_bio *rbio)
 {
-	return btrfs_alloc_page_array(rbio->nr_pages, rbio->stripe_pages);
+	int ret;
+
+	ret = btrfs_alloc_page_array(rbio->nr_pages, rbio->stripe_pages);
+	if (ret < 0)
+		return ret;
+	/* Mapping all sectors */
+	index_stripe_sectors(rbio);
+	return 0;
 }
 
 /* only allocate pages for p/q stripes */
 static int alloc_rbio_parity_pages(struct btrfs_raid_bio *rbio)
 {
 	int data_pages = rbio_stripe_page_index(rbio, rbio->nr_data, 0);
+	int ret;
 
-	return btrfs_alloc_page_array(rbio->nr_pages - data_pages,
-				      rbio->stripe_pages + data_pages);
+	ret = btrfs_alloc_page_array(rbio->nr_pages - data_pages,
+				     rbio->stripe_pages + data_pages);
+	if (ret < 0)
+		return ret;
+
+	index_stripe_sectors(rbio);
+	return 0;
 }
 
 /*
@@ -2286,6 +2340,7 @@ static int alloc_rbio_essential_pages(struct btrfs_raid_bio *rbio)
 			rbio->stripe_pages[index] = page;
 		}
 	}
+	index_stripe_sectors(rbio);
 	return 0;
 }
 
Signature · 1 line
2.35.1
author
#9 lore raw

PATCH v2 6/17 btrfs: introduce btrfs_raid_bio::bio_sectors

In reply to Qu Wenruo

This new member is going to fully replace bio_pages in the future, but for now let's keep them co-exist, until the full switch is done.

Currently cache_rbio_pages() and index_rbio_pages() will also populate the new array.

And cache_rbio_pages() need to record which sectors are uptodate, so we also need to introduce sector_ptr::uptodate bit.

To avoid extra memory usage, we let the new @uptodate bit to share bits with @pgoff.
Now pgoff only have at most 31 bits, which is already more than enough, as even for 256K page size, we only need 18 bits.

---
 fs/btrfs/raid56.c | 51 +++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)
fs/btrfs/raid56.c +49 -2
index 0c7b6a06d5ce..f88dd11a208d 100644
@@ -59,7 +59,8 @@ struct btrfs_stripe_hash_table {
  */
 struct sector_ptr {
 	struct page *page;
-	unsigned int pgoff;
+	unsigned int pgoff:24;
+	unsigned int uptodate:8;
 };
 
 enum btrfs_rbio_ops {
@@ -174,6 +175,9 @@ struct btrfs_raid_bio {
 	 */
 	struct page **stripe_pages;
 
+	/* Pointers to the sectors in the bio_list, for faster lookup */
+	struct sector_ptr *bio_sectors;
+
 	/*
 	 * pointers to the pages in the bio_list.  Stored
 	 * here for faster lookup
@@ -284,6 +288,20 @@ static void cache_rbio_pages(struct btrfs_raid_bio *rbio)
 		copy_highpage(rbio->stripe_pages[i], rbio->bio_pages[i]);
 		SetPageUptodate(rbio->stripe_pages[i]);
 	}
+
+	for (i = 0; i < rbio->nr_sectors; i++) {
+		/* Some range not covered by bio (partial write), skip it */
+		if (!rbio->bio_sectors[i].page)
+			continue;
+
+		ASSERT(rbio->stripe_sectors[i].page);
+		memcpy_page(rbio->stripe_sectors[i].page,
+			    rbio->stripe_sectors[i].pgoff,
+			    rbio->bio_sectors[i].page,
+			    rbio->bio_sectors[i].pgoff,
+			    rbio->bioc->fs_info->sectorsize);
+		rbio->stripe_sectors[i].uptodate = 1;
+	}
 	set_bit(RBIO_CACHE_READY_BIT, &rbio->flags);
 }
 
@@ -1013,7 +1031,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 
 	rbio = kzalloc(sizeof(*rbio) +
 		       sizeof(*rbio->stripe_pages) * num_pages +
-		       sizeof(*rbio->bio_pages) * num_pages +
+		       sizeof(*rbio->bio_sectors) * num_sectors +
 		       sizeof(*rbio->stripe_sectors) * num_sectors +
 		       sizeof(*rbio->finish_pointers) * real_stripes +
 		       sizeof(*rbio->dbitmap) * BITS_TO_LONGS(stripe_nsectors) +
@@ -1052,6 +1070,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	} while (0)
 	CONSUME_ALLOC(rbio->stripe_pages, num_pages);
 	CONSUME_ALLOC(rbio->bio_pages, num_pages);
+	CONSUME_ALLOC(rbio->bio_sectors, num_sectors);
 	CONSUME_ALLOC(rbio->stripe_sectors, num_sectors);
 	CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
 	CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_nsectors));
@@ -1171,6 +1190,31 @@ static void validate_rbio_for_rmw(struct btrfs_raid_bio *rbio)
 	}
 }
 
+static void index_one_bio(struct btrfs_raid_bio *rbio, struct bio *bio)
+{
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
+	struct bio_vec bvec;
+	struct bvec_iter iter;
+	u32 offset = (bio->bi_iter.bi_sector << SECTOR_SHIFT) -
+		     rbio->bioc->raid_map[0];
+
+	if (bio_flagged(bio, BIO_CLONED))
+		bio->bi_iter = btrfs_bio(bio)->iter;
+	bio_for_each_segment(bvec, bio, iter) {
+		u32 bvec_offset;
+
+		for (bvec_offset = 0; bvec_offset < bvec.bv_len;
+		     bvec_offset += sectorsize, offset += sectorsize) {
+			int index = offset / sectorsize;
+			struct sector_ptr *sector = &rbio->bio_sectors[index];
+
+			sector->page = bvec.bv_page;
+			sector->pgoff = bvec.bv_offset + bvec_offset;
+			ASSERT(sector->pgoff < PAGE_SIZE);
+		}
+	}
+}
+
 /*
  * helper function to walk our bio list and populate the bio_pages array with
  * the result.  This seems expensive, but it is faster than constantly
@@ -1201,6 +1245,9 @@ static void index_rbio_pages(struct btrfs_raid_bio *rbio)
 			i++;
 		}
 	}
+	bio_list_for_each(bio, &rbio->bio_list)
+		index_one_bio(rbio, bio);
+
 	spin_unlock_irq(&rbio->bio_list_lock);
 }
 
Signature · 1 line
2.35.1
author
#10 lore raw

PATCH v2 7/17 btrfs: make rbio_add_io_page() subpage compatible

In reply to Qu Wenruo

This involves:

- Rename rbio_add_io_page() to rbio_add_io_sector() Since we are still relying on PAGE_SIZE == sectorsize, add a new ASSERT() inside rbio_add_io_sector() to makesure all pgoff is 0.

- Introduce rbio_stripe_sector() helper
The equivalent of rbio_stripe_page().

This new helper has extra ASSERT()s to validate the stripe and sector number.

- Introduce sector_in_rbio() helper
The equivalent of page_in_rbio().

- Rename @pagenr variables to @sectornr

- Use rbio::stripe_nsectors when iterating the bitmap

Please note that, this only changes the interface, the bios are still using full page for IO.

---
 fs/btrfs/raid56.c | 247 ++++++++++++++++++++++++++++++----------------
 1 file changed, 161 insertions(+), 86 deletions(-)
fs/btrfs/raid56.c +161 -86
index f88dd11a208d..9c3e0201d032 100644
@@ -662,6 +662,25 @@ static int rbio_can_merge(struct btrfs_raid_bio *last,
 	return 1;
 }
 
+static unsigned int rbio_stripe_sector_index(const struct btrfs_raid_bio *rbio,
+					     unsigned int stripe_nr,
+					     unsigned int sector_nr)
+{
+	ASSERT(stripe_nr < rbio->real_stripes);
+	ASSERT(sector_nr < rbio->stripe_nsectors);
+
+	return stripe_nr * rbio->stripe_nsectors + sector_nr;
+}
+
+/* Return a sector from rbio->stripe_sectors, not from the bio list */
+static struct sector_ptr *rbio_stripe_sector(const struct btrfs_raid_bio *rbio,
+					     unsigned int stripe_nr,
+					     unsigned int sector_nr)
+{
+	return &rbio->stripe_sectors[rbio_stripe_sector_index(rbio, stripe_nr,
+							      sector_nr)];
+}
+
 static int rbio_stripe_page_index(struct btrfs_raid_bio *rbio, int stripe,
 				  int index)
 {
@@ -973,6 +992,44 @@ static void raid_write_end_io(struct bio *bio)
 	rbio_orig_end_io(rbio, err);
 }
 
+/*
+ * To get a sector pointer specified by its @stripe_nr and @sector_nr
+ *
+ * @stripe_nr:		Stripe number, valid range [0, real_stripe)
+ * @sector_nr:		Sector number inside the stripe,
+ *			valid range [0, stripe_nsectors)
+ * @bio_list_only:	Whether to use sectors inside the bio list only.
+ *
+ * The read/modify/write code wants to reuse the original bio page as much
+ * as possible, and only use stripe_sectors as fallback.
+ */
+static struct sector_ptr *sector_in_rbio(struct btrfs_raid_bio *rbio,
+					 int stripe_nr, int sector_nr,
+					 bool bio_list_only)
+{
+	struct sector_ptr *sector = NULL;
+	int index;
+
+	ASSERT(stripe_nr >= 0 && stripe_nr < rbio->real_stripes);
+	ASSERT(sector_nr >= 0 && sector_nr < rbio->stripe_nsectors);
+
+	index = stripe_nr * rbio->stripe_nsectors + sector_nr;
+	ASSERT(index >= 0 && index < rbio->nr_sectors);
+
+	spin_lock_irq(&rbio->bio_list_lock);
+	sector = &rbio->bio_sectors[index];
+	if (sector->page || bio_list_only) {
+		/* Don't return sector without a valid page pointer */
+		if (!sector->page)
+			sector = NULL;
+		spin_unlock_irq(&rbio->bio_list_lock);
+		return sector;
+	}
+	spin_unlock_irq(&rbio->bio_list_lock);
+
+	return &rbio->stripe_sectors[index];
+}
+
 /*
  * the read/modify/write code wants to use the original bio for
  * any pages it included, and then use the rbio for everything
@@ -1120,26 +1177,39 @@ static int alloc_rbio_parity_pages(struct btrfs_raid_bio *rbio)
 }
 
 /*
- * add a single page from a specific stripe into our list of bios for IO
- * this will try to merge into existing bios if possible, and returns
- * zero if all went well.
+ * Add a single sector @sector into our list of bios for IO.
+ *
+ * Return 0 if everything went well.
+ * Return <0 for error.
  */
-static int rbio_add_io_page(struct btrfs_raid_bio *rbio,
-			    struct bio_list *bio_list,
-			    struct page *page,
-			    int stripe_nr,
-			    unsigned long page_index,
-			    unsigned long bio_max_len,
-			    unsigned int opf)
+static int rbio_add_io_sector(struct btrfs_raid_bio *rbio,
+			      struct bio_list *bio_list,
+			      struct sector_ptr *sector,
+			      unsigned int stripe_nr,
+			      unsigned int sector_nr,
+			      unsigned long bio_max_len, unsigned int opf)
 {
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
 	struct bio *last = bio_list->tail;
 	int ret;
 	struct bio *bio;
 	struct btrfs_io_stripe *stripe;
 	u64 disk_start;
 
+	/*
+	 * NOTE: here stripe_nr has taken device replace into consideration,
+	 * thus it can be larger than rbio->real_stripe.
+	 * So here we check against bioc->num_stripes, not rbio->real_stripes.
+	 */
+	ASSERT(stripe_nr >= 0 && stripe_nr < rbio->bioc->num_stripes);
+	ASSERT(sector_nr >= 0 && sector_nr < rbio->stripe_nsectors);
+	ASSERT(sector->page);
+
+	/* We don't yet support subpage, thus pgoff should always be 0 */
+	ASSERT(sector->pgoff == 0);
+
 	stripe = &rbio->bioc->stripes[stripe_nr];
-	disk_start = stripe->physical + (page_index << PAGE_SHIFT);
+	disk_start = stripe->physical + sector_nr * sectorsize;
 
 	/* if the device is missing, just fail this stripe */
 	if (!stripe->dev->bdev)
@@ -1156,8 +1226,9 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,
 		 */
 		if (last_end == disk_start && !last->bi_status &&
 		    last->bi_bdev == stripe->dev->bdev) {
-			ret = bio_add_page(last, page, PAGE_SIZE, 0);
-			if (ret == PAGE_SIZE)
+			ret = bio_add_page(last, sector->page, sectorsize,
+					   sector->pgoff);
+			if (ret == sectorsize)
 				return 0;
 		}
 	}
@@ -1168,7 +1239,7 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,
 	bio->bi_iter.bi_sector = disk_start >> 9;
 	bio->bi_private = rbio;
 
-	bio_add_page(bio, page, PAGE_SIZE, 0);
+	bio_add_page(bio, sector->page, sectorsize, sector->pgoff);
 	bio_list_add(bio_list, bio);
 	return 0;
 }
@@ -1265,7 +1336,7 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 	void **pointers = rbio->finish_pointers;
 	int nr_data = rbio->nr_data;
 	int stripe;
-	int pagenr;
+	int sectornr;
 	bool has_qstripe;
 	struct bio_list bio_list;
 	struct bio *bio;
@@ -1309,16 +1380,16 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 	else
 		clear_bit(RBIO_CACHE_READY_BIT, &rbio->flags);
 
-	for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
+	for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
 		struct page *p;
 		/* first collect one page from each data stripe */
 		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, pagenr, 0);
+			p = page_in_rbio(rbio, stripe, sectornr, 0);
 			pointers[stripe] = kmap_local_page(p);
 		}
 
 		/* then add the parity stripe */
-		p = rbio_pstripe_page(rbio, pagenr);
+		p = rbio_pstripe_page(rbio, sectornr);
 		SetPageUptodate(p);
 		pointers[stripe++] = kmap_local_page(p);
 
@@ -1328,7 +1399,7 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 			 * raid6, add the qstripe and call the
 			 * library function to fill in our p/q
 			 */
-			p = rbio_qstripe_page(rbio, pagenr);
+			p = rbio_qstripe_page(rbio, sectornr);
 			SetPageUptodate(p);
 			pointers[stripe++] = kmap_local_page(p);
 
@@ -1349,19 +1420,19 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 	 * everything else.
 	 */
 	for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
 			if (stripe < rbio->nr_data) {
-				page = page_in_rbio(rbio, stripe, pagenr, 1);
-				if (!page)
+				sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+				if (!sector)
 					continue;
 			} else {
-			       page = rbio_stripe_page(rbio, stripe, pagenr);
+				sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			}
 
-			ret = rbio_add_io_page(rbio, &bio_list,
-				       page, stripe, pagenr, rbio->stripe_len,
-				       REQ_OP_WRITE);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector, stripe,
+						 sectornr, rbio->stripe_len, REQ_OP_WRITE);
 			if (ret)
 				goto cleanup;
 		}
@@ -1374,20 +1445,21 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 		if (!bioc->tgtdev_map[stripe])
 			continue;
 
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
 			if (stripe < rbio->nr_data) {
-				page = page_in_rbio(rbio, stripe, pagenr, 1);
-				if (!page)
+				sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+				if (!sector)
 					continue;
 			} else {
-			       page = rbio_stripe_page(rbio, stripe, pagenr);
+				sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			}
 
-			ret = rbio_add_io_page(rbio, &bio_list, page,
-					       rbio->bioc->tgtdev_map[stripe],
-					       pagenr, rbio->stripe_len,
-					       REQ_OP_WRITE);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 rbio->bioc->tgtdev_map[stripe],
+						 sectornr, rbio->stripe_len,
+						 REQ_OP_WRITE);
 			if (ret)
 				goto cleanup;
 		}
@@ -1563,7 +1635,7 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)
 	int bios_to_read = 0;
 	struct bio_list bio_list;
 	int ret;
-	int pagenr;
+	int sectornr;
 	int stripe;
 	struct bio *bio;
 
@@ -1581,28 +1653,29 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)
 	 * stripe
 	 */
 	for (stripe = 0; stripe < rbio->nr_data; stripe++) {
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
 			/*
-			 * we want to find all the pages missing from
+			 * We want to find all the sectors missing from
 			 * the rbio and read them from the disk.  If
-			 * page_in_rbio finds a page in the bio list
+			 * sector_in_rbio() finds a page in the bio list
 			 * we don't need to read it off the stripe.
 			 */
-			page = page_in_rbio(rbio, stripe, pagenr, 1);
-			if (page)
+			sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+			if (sector)
 				continue;
 
-			page = rbio_stripe_page(rbio, stripe, pagenr);
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			/*
-			 * the bio cache may have handed us an uptodate
+			 * The bio cache may have handed us an uptodate
 			 * page.  If so, be happy and use it
 			 */
-			if (PageUptodate(page))
+			if (sector->uptodate)
 				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list, page,
-				       stripe, pagenr, rbio->stripe_len,
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+				       stripe, sectornr, rbio->stripe_len,
 				       REQ_OP_READ);
 			if (ret)
 				goto cleanup;
@@ -2107,7 +2180,7 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)
 	int bios_to_read = 0;
 	struct bio_list bio_list;
 	int ret;
-	int pagenr;
+	int sectornr;
 	int stripe;
 	struct bio *bio;
 
@@ -2130,21 +2203,20 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)
 			continue;
 		}
 
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *p;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
 
 			/*
 			 * the rmw code may have already read this
 			 * page in
 			 */
-			p = rbio_stripe_page(rbio, stripe, pagenr);
-			if (PageUptodate(p))
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
+			if (sector->uptodate)
 				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list,
-				       rbio_stripe_page(rbio, stripe, pagenr),
-				       stripe, pagenr, rbio->stripe_len,
-				       REQ_OP_READ);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 stripe, sectornr,
+						 rbio->stripe_len, REQ_OP_READ);
 			if (ret < 0)
 				goto cleanup;
 		}
@@ -2399,7 +2471,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	unsigned long *pbitmap = rbio->finish_pbitmap;
 	int nr_data = rbio->nr_data;
 	int stripe;
-	int pagenr;
+	int sectornr;
 	bool has_qstripe;
 	struct page *p_page = NULL;
 	struct page *q_page = NULL;
@@ -2419,7 +2491,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 
 	if (bioc->num_tgtdevs && bioc->tgtdev_map[rbio->scrubp]) {
 		is_replace = 1;
-		bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_npages);
+		bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_nsectors);
 	}
 
 	/*
@@ -2453,12 +2525,12 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	/* Map the parity stripe just once */
 	pointers[nr_data] = kmap_local_page(p_page);
 
-	for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
+	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
 		struct page *p;
 		void *parity;
 		/* first collect one page from each data stripe */
 		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, pagenr, 0);
+			p = page_in_rbio(rbio, stripe, sectornr, 0);
 			pointers[stripe] = kmap_local_page(p);
 		}
 
@@ -2473,13 +2545,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 		}
 
 		/* Check scrubbing parity and repair it */
-		p = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
+		p = rbio_stripe_page(rbio, rbio->scrubp, sectornr);
 		parity = kmap_local_page(p);
 		if (memcmp(parity, pointers[rbio->scrubp], PAGE_SIZE))
 			copy_page(parity, pointers[rbio->scrubp]);
 		else
 			/* Parity is right, needn't writeback */
-			bitmap_clear(rbio->dbitmap, pagenr, 1);
+			bitmap_clear(rbio->dbitmap, sectornr, 1);
 		kunmap_local(parity);
 
 		for (stripe = nr_data - 1; stripe >= 0; stripe--)
@@ -2499,12 +2571,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	 * higher layers (the bio_list in our rbio) and our p/q.  Ignore
 	 * everything else.
 	 */
-	for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-		struct page *page;
+	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
+		struct sector_ptr *sector;
 
-		page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-		ret = rbio_add_io_page(rbio, &bio_list, page, rbio->scrubp,
-				       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+		sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+		ret = rbio_add_io_sector(rbio, &bio_list, sector, rbio->scrubp,
+					 sectornr, rbio->stripe_len,
+					 REQ_OP_WRITE);
 		if (ret)
 			goto cleanup;
 	}
@@ -2512,13 +2585,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	if (!is_replace)
 		goto submit_write;
 
-	for_each_set_bit(pagenr, pbitmap, rbio->stripe_npages) {
-		struct page *page;
+	for_each_set_bit(sectornr, pbitmap, rbio->stripe_nsectors) {
+		struct sector_ptr *sector;
 
-		page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-		ret = rbio_add_io_page(rbio, &bio_list, page,
+		sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+		ret = rbio_add_io_sector(rbio, &bio_list, sector,
 				       bioc->tgtdev_map[rbio->scrubp],
-				       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+				       sectornr, rbio->stripe_len, REQ_OP_WRITE);
 		if (ret)
 			goto cleanup;
 	}
@@ -2650,7 +2723,7 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)
 	int bios_to_read = 0;
 	struct bio_list bio_list;
 	int ret;
-	int pagenr;
+	int sectornr;
 	int stripe;
 	struct bio *bio;
 
@@ -2666,28 +2739,30 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)
 	 * stripe
 	 */
 	for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-		for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-			struct page *page;
+		for_each_set_bit(sectornr, rbio->dbitmap,
+				 rbio->stripe_nsectors) {
+			struct sector_ptr *sector;
 			/*
-			 * we want to find all the pages missing from
+			 * We want to find all the sectors missing from
 			 * the rbio and read them from the disk.  If
-			 * page_in_rbio finds a page in the bio list
+			 * sector_in_rbio() finds a sector in the bio list
 			 * we don't need to read it off the stripe.
 			 */
-			page = page_in_rbio(rbio, stripe, pagenr, 1);
-			if (page)
+			sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+			if (sector)
 				continue;
 
-			page = rbio_stripe_page(rbio, stripe, pagenr);
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			/*
-			 * the bio cache may have handed us an uptodate
-			 * page.  If so, be happy and use it
+			 * The bio cache may have handed us an uptodate
+			 * sector.  If so, be happy and use it
 			 */
-			if (PageUptodate(page))
+			if (sector->uptodate)
 				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list, page, stripe,
-					       pagenr, rbio->stripe_len, REQ_OP_READ);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 stripe, sectornr,
+						 rbio->stripe_len, REQ_OP_READ);
 			if (ret)
 				goto cleanup;
 		}
Signature · 1 line
2.35.1
#11 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

There's an assertion failure reported by btrfs/023

Quoted 12 lines · Qu Wenruo
fs/btrfs/raid56.c +7 -0
+static int rbio_add_io_sector(struct btrfs_raid_bio *rbio,
+			      struct bio_list *bio_list,
+			      struct sector_ptr *sector,
+			      unsigned int stripe_nr,
+			      unsigned int sector_nr,
+			      unsigned long bio_max_len, unsigned int opf)
 {
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
 	struct bio *last = bio_list->tail;
 	int ret;
 	struct bio *bio;
 	struct btrfs_io_stripe *stripe;

fs/btrfs/raid56.c static int alloc_rbio_parity_pages(struct btrfs_raid_bio *rbio) · line 1206

Quoted 10 lines
fs/btrfs/raid56.c +8 -0
 	u64 disk_start;
 
+	/*
+	 * NOTE: here stripe_nr has taken device replace into consideration,
+	 * thus it can be larger than rbio->real_stripe.
+	 * So here we check against bioc->num_stripes, not rbio->real_stripes.
+	 */
+	ASSERT(stripe_nr >= 0 && stripe_nr < rbio->bioc->num_stripes);
+	ASSERT(sector_nr >= 0 && sector_nr < rbio->stripe_nsectors);
+	ASSERT(sector->page);

This one ^^^

[ 2280.705765] assertion failed: sector->page, in fs/btrfs/raid56.c:1145
[ 2280.707844] ------------[ cut here ]------------
[ 2280.709401] kernel BUG at fs/btrfs/ctree.h:3614!
[ 2280.711084] invalid opcode: 0000 [#1] PREEMPT SMP
[ 2280.712822] CPU: 3 PID: 4084 Comm: kworker/u8:2 Not tainted 5.18.0-rc2-default+ #1697
[ 2280.715648] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014
[ 2280.719656] Workqueue: btrfs-rmw btrfs_work_helper [btrfs]
[ 2280.721775] RIP: 0010:assertfail.constprop.0+0x18/0x1a [btrfs]
[ 2280.729575] RSP: 0018:ffffad6d071afda0 EFLAGS: 00010246
[ 2280.730844] RAX: 0000000000000039 RBX: 0000000000000000 RCX: 0000000000000000
[ 2280.732449] RDX: 0000000000000000 RSI: 0000000000000003 RDI: 00000000ffffffff
[ 2280.733992] RBP: ffff8e51d5465000 R08: 0000000000000003 R09: 0000000000000002
[ 2280.735535] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000003
[ 2280.737093] R13: ffff8e51d5465000 R14: ffff8e51d5465d78 R15: 0000000000001000
[ 2280.738613] FS:  0000000000000000(0000) GS:ffff8e523dc00000(0000) knlGS:0000000000000000
[ 2280.740392] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2280.741794] CR2: 000055f48fe51ab0 CR3: 000000001f805001 CR4: 0000000000170ea0
[ 2280.743532] Call Trace:
[ 2280.744136]  <TASK>
[ 2280.744701]  rbio_add_io_sector.cold+0x11/0x33 [btrfs]
[ 2280.745846]  ? _raw_spin_unlock_irq+0x2f/0x50
[ 2280.746782]  raid56_rmw_stripe.isra.0+0x153/0x320 [btrfs]
[ 2280.747965]  btrfs_work_helper+0xd6/0x1d0 [btrfs]
[ 2280.749018]  process_one_work+0x264/0x5f0
[ 2280.749806]  worker_thread+0x52/0x3b0
[ 2280.750523]  ? process_one_work+0x5f0/0x5f0
[ 2280.751395]  kthread+0xea/0x110
[ 2280.752097]  ? kthread_complete_and_exit+0x20/0x20
[ 2280.753112]  ret_from_fork+0x1f/0x30
Quoted 357 lines
fs/btrfs/raid56.c +85 -76
+
+	/* We don't yet support subpage, thus pgoff should always be 0 */
+	ASSERT(sector->pgoff == 0);
+
 	stripe = &rbio->bioc->stripes[stripe_nr];
-	disk_start = stripe->physical + (page_index << PAGE_SHIFT);
+	disk_start = stripe->physical + sector_nr * sectorsize;
 
 	/* if the device is missing, just fail this stripe */
 	if (!stripe->dev->bdev)
@@ -1156,8 +1226,9 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,
 		 */
 		if (last_end == disk_start && !last->bi_status &&
 		    last->bi_bdev == stripe->dev->bdev) {
-			ret = bio_add_page(last, page, PAGE_SIZE, 0);
-			if (ret == PAGE_SIZE)
+			ret = bio_add_page(last, sector->page, sectorsize,
+					   sector->pgoff);
+			if (ret == sectorsize)
 				return 0;
 		}
 	}
@@ -1168,7 +1239,7 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,
 	bio->bi_iter.bi_sector = disk_start >> 9;
 	bio->bi_private = rbio;
 
-	bio_add_page(bio, page, PAGE_SIZE, 0);
+	bio_add_page(bio, sector->page, sectorsize, sector->pgoff);
 	bio_list_add(bio_list, bio);
 	return 0;
 }
@@ -1265,7 +1336,7 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 	void **pointers = rbio->finish_pointers;
 	int nr_data = rbio->nr_data;
 	int stripe;
-	int pagenr;
+	int sectornr;
 	bool has_qstripe;
 	struct bio_list bio_list;
 	struct bio *bio;
@@ -1309,16 +1380,16 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 	else
 		clear_bit(RBIO_CACHE_READY_BIT, &rbio->flags);
 
-	for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
+	for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
 		struct page *p;
 		/* first collect one page from each data stripe */
 		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, pagenr, 0);
+			p = page_in_rbio(rbio, stripe, sectornr, 0);
 			pointers[stripe] = kmap_local_page(p);
 		}
 
 		/* then add the parity stripe */
-		p = rbio_pstripe_page(rbio, pagenr);
+		p = rbio_pstripe_page(rbio, sectornr);
 		SetPageUptodate(p);
 		pointers[stripe++] = kmap_local_page(p);
 
@@ -1328,7 +1399,7 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 			 * raid6, add the qstripe and call the
 			 * library function to fill in our p/q
 			 */
-			p = rbio_qstripe_page(rbio, pagenr);
+			p = rbio_qstripe_page(rbio, sectornr);
 			SetPageUptodate(p);
 			pointers[stripe++] = kmap_local_page(p);
 
@@ -1349,19 +1420,19 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 	 * everything else.
 	 */
 	for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
 			if (stripe < rbio->nr_data) {
-				page = page_in_rbio(rbio, stripe, pagenr, 1);
-				if (!page)
+				sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+				if (!sector)
 					continue;
 			} else {
-			       page = rbio_stripe_page(rbio, stripe, pagenr);
+				sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			}
 
-			ret = rbio_add_io_page(rbio, &bio_list,
-				       page, stripe, pagenr, rbio->stripe_len,
-				       REQ_OP_WRITE);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector, stripe,
+						 sectornr, rbio->stripe_len, REQ_OP_WRITE);
 			if (ret)
 				goto cleanup;
 		}
@@ -1374,20 +1445,21 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 		if (!bioc->tgtdev_map[stripe])
 			continue;
 
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
 			if (stripe < rbio->nr_data) {
-				page = page_in_rbio(rbio, stripe, pagenr, 1);
-				if (!page)
+				sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+				if (!sector)
 					continue;
 			} else {
-			       page = rbio_stripe_page(rbio, stripe, pagenr);
+				sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			}
 
-			ret = rbio_add_io_page(rbio, &bio_list, page,
-					       rbio->bioc->tgtdev_map[stripe],
-					       pagenr, rbio->stripe_len,
-					       REQ_OP_WRITE);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 rbio->bioc->tgtdev_map[stripe],
+						 sectornr, rbio->stripe_len,
+						 REQ_OP_WRITE);
 			if (ret)
 				goto cleanup;
 		}
@@ -1563,7 +1635,7 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)
 	int bios_to_read = 0;
 	struct bio_list bio_list;
 	int ret;
-	int pagenr;
+	int sectornr;
 	int stripe;
 	struct bio *bio;
 
@@ -1581,28 +1653,29 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)
 	 * stripe
 	 */
 	for (stripe = 0; stripe < rbio->nr_data; stripe++) {
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
 			/*
-			 * we want to find all the pages missing from
+			 * We want to find all the sectors missing from
 			 * the rbio and read them from the disk.  If
-			 * page_in_rbio finds a page in the bio list
+			 * sector_in_rbio() finds a page in the bio list
 			 * we don't need to read it off the stripe.
 			 */
-			page = page_in_rbio(rbio, stripe, pagenr, 1);
-			if (page)
+			sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+			if (sector)
 				continue;
 
-			page = rbio_stripe_page(rbio, stripe, pagenr);
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			/*
-			 * the bio cache may have handed us an uptodate
+			 * The bio cache may have handed us an uptodate
 			 * page.  If so, be happy and use it
 			 */
-			if (PageUptodate(page))
+			if (sector->uptodate)
 				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list, page,
-				       stripe, pagenr, rbio->stripe_len,
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+				       stripe, sectornr, rbio->stripe_len,
 				       REQ_OP_READ);
 			if (ret)
 				goto cleanup;
@@ -2107,7 +2180,7 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)
 	int bios_to_read = 0;
 	struct bio_list bio_list;
 	int ret;
-	int pagenr;
+	int sectornr;
 	int stripe;
 	struct bio *bio;
 
@@ -2130,21 +2203,20 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)
 			continue;
 		}
 
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *p;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
 
 			/*
 			 * the rmw code may have already read this
 			 * page in
 			 */
-			p = rbio_stripe_page(rbio, stripe, pagenr);
-			if (PageUptodate(p))
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
+			if (sector->uptodate)
 				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list,
-				       rbio_stripe_page(rbio, stripe, pagenr),
-				       stripe, pagenr, rbio->stripe_len,
-				       REQ_OP_READ);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 stripe, sectornr,
+						 rbio->stripe_len, REQ_OP_READ);
 			if (ret < 0)
 				goto cleanup;
 		}
@@ -2399,7 +2471,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	unsigned long *pbitmap = rbio->finish_pbitmap;
 	int nr_data = rbio->nr_data;
 	int stripe;
-	int pagenr;
+	int sectornr;
 	bool has_qstripe;
 	struct page *p_page = NULL;
 	struct page *q_page = NULL;
@@ -2419,7 +2491,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 
 	if (bioc->num_tgtdevs && bioc->tgtdev_map[rbio->scrubp]) {
 		is_replace = 1;
-		bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_npages);
+		bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_nsectors);
 	}
 
 	/*
@@ -2453,12 +2525,12 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	/* Map the parity stripe just once */
 	pointers[nr_data] = kmap_local_page(p_page);
 
-	for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
+	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
 		struct page *p;
 		void *parity;
 		/* first collect one page from each data stripe */
 		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, pagenr, 0);
+			p = page_in_rbio(rbio, stripe, sectornr, 0);
 			pointers[stripe] = kmap_local_page(p);
 		}
 
@@ -2473,13 +2545,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 		}
 
 		/* Check scrubbing parity and repair it */
-		p = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
+		p = rbio_stripe_page(rbio, rbio->scrubp, sectornr);
 		parity = kmap_local_page(p);
 		if (memcmp(parity, pointers[rbio->scrubp], PAGE_SIZE))
 			copy_page(parity, pointers[rbio->scrubp]);
 		else
 			/* Parity is right, needn't writeback */
-			bitmap_clear(rbio->dbitmap, pagenr, 1);
+			bitmap_clear(rbio->dbitmap, sectornr, 1);
 		kunmap_local(parity);
 
 		for (stripe = nr_data - 1; stripe >= 0; stripe--)
@@ -2499,12 +2571,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	 * higher layers (the bio_list in our rbio) and our p/q.  Ignore
 	 * everything else.
 	 */
-	for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-		struct page *page;
+	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
+		struct sector_ptr *sector;
 
-		page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-		ret = rbio_add_io_page(rbio, &bio_list, page, rbio->scrubp,
-				       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+		sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+		ret = rbio_add_io_sector(rbio, &bio_list, sector, rbio->scrubp,
+					 sectornr, rbio->stripe_len,
+					 REQ_OP_WRITE);
 		if (ret)
 			goto cleanup;
 	}
@@ -2512,13 +2585,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	if (!is_replace)
 		goto submit_write;
 
-	for_each_set_bit(pagenr, pbitmap, rbio->stripe_npages) {
-		struct page *page;
+	for_each_set_bit(sectornr, pbitmap, rbio->stripe_nsectors) {
+		struct sector_ptr *sector;
 
-		page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-		ret = rbio_add_io_page(rbio, &bio_list, page,
+		sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+		ret = rbio_add_io_sector(rbio, &bio_list, sector,
 				       bioc->tgtdev_map[rbio->scrubp],
-				       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+				       sectornr, rbio->stripe_len, REQ_OP_WRITE);
 		if (ret)
 			goto cleanup;
 	}
@@ -2650,7 +2723,7 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)
 	int bios_to_read = 0;
 	struct bio_list bio_list;
 	int ret;
-	int pagenr;
+	int sectornr;
 	int stripe;
 	struct bio *bio;
 
@@ -2666,28 +2739,30 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)
 	 * stripe
 	 */
 	for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-		for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-			struct page *page;
+		for_each_set_bit(sectornr, rbio->dbitmap,
+				 rbio->stripe_nsectors) {
+			struct sector_ptr *sector;
 			/*
-			 * we want to find all the pages missing from
+			 * We want to find all the sectors missing from
 			 * the rbio and read them from the disk.  If
-			 * page_in_rbio finds a page in the bio list
+			 * sector_in_rbio() finds a sector in the bio list
 			 * we don't need to read it off the stripe.
 			 */
-			page = page_in_rbio(rbio, stripe, pagenr, 1);
-			if (page)
+			sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+			if (sector)
 				continue;
 
-			page = rbio_stripe_page(rbio, stripe, pagenr);
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			/*
-			 * the bio cache may have handed us an uptodate
-			 * page.  If so, be happy and use it
+			 * The bio cache may have handed us an uptodate
+			 * sector.  If so, be happy and use it
 			 */
-			if (PageUptodate(page))
+			if (sector->uptodate)
 				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list, page, stripe,
-					       pagenr, rbio->stripe_len, REQ_OP_READ);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 stripe, sectornr,
+						 rbio->stripe_len, REQ_OP_READ);
 			if (ret)
 				goto cleanup;
 		}
Signature · 1 line
2.35.1
#12 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 17 lines · David Sterba

There's an assertion failure reported by btrfs/023

Quoted 14 lines · Qu Wenruo
fs/btrfs/raid56.c +7 -0
+static int rbio_add_io_sector(struct btrfs_raid_bio *rbio,
+			      struct bio_list *bio_list,
+			      struct sector_ptr *sector,
+			      unsigned int stripe_nr,
+			      unsigned int sector_nr,
+			      unsigned long bio_max_len, unsigned int opf)
  {
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
  	struct bio *last = bio_list->tail;
  	int ret;
  	struct bio *bio;
  	struct btrfs_io_stripe *stripe;
  	u64 disk_start;
Quoted 10 lines

fs/btrfs/raid56.c static int alloc_rbio_parity_pages(struct btrfs_raid_bio *rbio) · line 1206

Quoted 8 lines
fs/btrfs/raid56.c +8 -0
+	/*
+	 * NOTE: here stripe_nr has taken device replace into consideration,
+	 * thus it can be larger than rbio->real_stripe.
+	 * So here we check against bioc->num_stripes, not rbio->real_stripes.
+	 */
+	ASSERT(stripe_nr >= 0 && stripe_nr < rbio->bioc->num_stripes);
+	ASSERT(sector_nr >= 0 && sector_nr < rbio->stripe_nsectors);
+	ASSERT(sector->page);

This one ^^^

Failed to reproduce here, both x86_64 and aarch64 survived over 64 loops of btrfs/023.

Although that's withy my github branch. Let me check the current branch to see what's wrong.

Thanks,
Qu

Quoted 389 lines
[ 2280.705765] assertion failed: sector->page, in fs/btrfs/raid56.c:1145
[ 2280.707844] ------------[ cut here ]------------
[ 2280.709401] kernel BUG at fs/btrfs/ctree.h:3614!
[ 2280.711084] invalid opcode: 0000 [#1] PREEMPT SMP
[ 2280.712822] CPU: 3 PID: 4084 Comm: kworker/u8:2 Not tainted 5.18.0-rc2-default+ #1697
[ 2280.715648] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014
[ 2280.719656] Workqueue: btrfs-rmw btrfs_work_helper [btrfs]
[ 2280.721775] RIP: 0010:assertfail.constprop.0+0x18/0x1a [btrfs]
[ 2280.729575] RSP: 0018:ffffad6d071afda0 EFLAGS: 00010246
[ 2280.730844] RAX: 0000000000000039 RBX: 0000000000000000 RCX: 0000000000000000
[ 2280.732449] RDX: 0000000000000000 RSI: 0000000000000003 RDI: 00000000ffffffff
[ 2280.733992] RBP: ffff8e51d5465000 R08: 0000000000000003 R09: 0000000000000002
[ 2280.735535] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000003
[ 2280.737093] R13: ffff8e51d5465000 R14: ffff8e51d5465d78 R15: 0000000000001000
[ 2280.738613] FS:  0000000000000000(0000) GS:ffff8e523dc00000(0000) knlGS:0000000000000000
[ 2280.740392] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2280.741794] CR2: 000055f48fe51ab0 CR3: 000000001f805001 CR4: 0000000000170ea0
[ 2280.743532] Call Trace:
[ 2280.744136]  <TASK>
[ 2280.744701]  rbio_add_io_sector.cold+0x11/0x33 [btrfs]
[ 2280.745846]  ? _raw_spin_unlock_irq+0x2f/0x50
[ 2280.746782]  raid56_rmw_stripe.isra.0+0x153/0x320 [btrfs]
[ 2280.747965]  btrfs_work_helper+0xd6/0x1d0 [btrfs]
[ 2280.749018]  process_one_work+0x264/0x5f0
[ 2280.749806]  worker_thread+0x52/0x3b0
[ 2280.750523]  ? process_one_work+0x5f0/0x5f0
[ 2280.751395]  kthread+0xea/0x110
[ 2280.752097]  ? kthread_complete_and_exit+0x20/0x20
[ 2280.753112]  ret_from_fork+0x1f/0x30
Quoted 357 lines
fs/btrfs/raid56.c +85 -77
+
+	/* We don't yet support subpage, thus pgoff should always be 0 */
+	ASSERT(sector->pgoff == 0);
+
  	stripe = &rbio->bioc->stripes[stripe_nr];
-	disk_start = stripe->physical + (page_index << PAGE_SHIFT);
+	disk_start = stripe->physical + sector_nr * sectorsize;
 
  	/* if the device is missing, just fail this stripe */
  	if (!stripe->dev->bdev)
@@ -1156,8 +1226,9 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,
  		 */
  		if (last_end == disk_start && !last->bi_status &&
  		    last->bi_bdev == stripe->dev->bdev) {
-			ret = bio_add_page(last, page, PAGE_SIZE, 0);
-			if (ret == PAGE_SIZE)
+			ret = bio_add_page(last, sector->page, sectorsize,
+					   sector->pgoff);
+			if (ret == sectorsize)
  				return 0;
  		}
  	}
@@ -1168,7 +1239,7 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,
  	bio->bi_iter.bi_sector = disk_start >> 9;
  	bio->bi_private = rbio;
 
-	bio_add_page(bio, page, PAGE_SIZE, 0);
+	bio_add_page(bio, sector->page, sectorsize, sector->pgoff);
  	bio_list_add(bio_list, bio);
  	return 0;
  }
@@ -1265,7 +1336,7 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
  	void **pointers = rbio->finish_pointers;
  	int nr_data = rbio->nr_data;
  	int stripe;
-	int pagenr;
+	int sectornr;
  	bool has_qstripe;
  	struct bio_list bio_list;
  	struct bio *bio;
@@ -1309,16 +1380,16 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
  	else
  		clear_bit(RBIO_CACHE_READY_BIT, &rbio->flags);
 
-	for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
+	for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
  		struct page *p;
  		/* first collect one page from each data stripe */
  		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, pagenr, 0);
+			p = page_in_rbio(rbio, stripe, sectornr, 0);
  			pointers[stripe] = kmap_local_page(p);
  		}
 
  		/* then add the parity stripe */
-		p = rbio_pstripe_page(rbio, pagenr);
+		p = rbio_pstripe_page(rbio, sectornr);
  		SetPageUptodate(p);
  		pointers[stripe++] = kmap_local_page(p);
 
@@ -1328,7 +1399,7 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
  			 * raid6, add the qstripe and call the
  			 * library function to fill in our p/q
  			 */
-			p = rbio_qstripe_page(rbio, pagenr);
+			p = rbio_qstripe_page(rbio, sectornr);
  			SetPageUptodate(p);
  			pointers[stripe++] = kmap_local_page(p);
 
@@ -1349,19 +1420,19 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
  	 * everything else.
  	 */
  	for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
  			if (stripe < rbio->nr_data) {
-				page = page_in_rbio(rbio, stripe, pagenr, 1);
-				if (!page)
+				sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+				if (!sector)
  					continue;
  			} else {
-			       page = rbio_stripe_page(rbio, stripe, pagenr);
+				sector = rbio_stripe_sector(rbio, stripe, sectornr);
  			}
 
-			ret = rbio_add_io_page(rbio, &bio_list,
-				       page, stripe, pagenr, rbio->stripe_len,
-				       REQ_OP_WRITE);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector, stripe,
+						 sectornr, rbio->stripe_len, REQ_OP_WRITE);
  			if (ret)
  				goto cleanup;
  		}
@@ -1374,20 +1445,21 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
  		if (!bioc->tgtdev_map[stripe])
  			continue;
 
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
  			if (stripe < rbio->nr_data) {
-				page = page_in_rbio(rbio, stripe, pagenr, 1);
-				if (!page)
+				sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+				if (!sector)
  					continue;
  			} else {
-			       page = rbio_stripe_page(rbio, stripe, pagenr);
+				sector = rbio_stripe_sector(rbio, stripe, sectornr);
  			}
 
-			ret = rbio_add_io_page(rbio, &bio_list, page,
-					       rbio->bioc->tgtdev_map[stripe],
-					       pagenr, rbio->stripe_len,
-					       REQ_OP_WRITE);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 rbio->bioc->tgtdev_map[stripe],
+						 sectornr, rbio->stripe_len,
+						 REQ_OP_WRITE);
  			if (ret)
  				goto cleanup;
  		}
@@ -1563,7 +1635,7 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)
  	int bios_to_read = 0;
  	struct bio_list bio_list;
  	int ret;
-	int pagenr;
+	int sectornr;
  	int stripe;
  	struct bio *bio;
 
@@ -1581,28 +1653,29 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)
  	 * stripe
  	 */
  	for (stripe = 0; stripe < rbio->nr_data; stripe++) {
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *page;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
+
  			/*
-			 * we want to find all the pages missing from
+			 * We want to find all the sectors missing from
  			 * the rbio and read them from the disk.  If
-			 * page_in_rbio finds a page in the bio list
+			 * sector_in_rbio() finds a page in the bio list
  			 * we don't need to read it off the stripe.
  			 */
-			page = page_in_rbio(rbio, stripe, pagenr, 1);
-			if (page)
+			sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+			if (sector)
  				continue;
 
-			page = rbio_stripe_page(rbio, stripe, pagenr);
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
  			/*
-			 * the bio cache may have handed us an uptodate
+			 * The bio cache may have handed us an uptodate
  			 * page.  If so, be happy and use it
  			 */
-			if (PageUptodate(page))
+			if (sector->uptodate)
  				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list, page,
-				       stripe, pagenr, rbio->stripe_len,
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+				       stripe, sectornr, rbio->stripe_len,
  				       REQ_OP_READ);
  			if (ret)
  				goto cleanup;
@@ -2107,7 +2180,7 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)
  	int bios_to_read = 0;
  	struct bio_list bio_list;
  	int ret;
-	int pagenr;
+	int sectornr;
  	int stripe;
  	struct bio *bio;
 
@@ -2130,21 +2203,20 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)
  			continue;
  		}
 
-		for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-			struct page *p;
+		for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+			struct sector_ptr *sector;
 
  			/*
  			 * the rmw code may have already read this
  			 * page in
  			 */
-			p = rbio_stripe_page(rbio, stripe, pagenr);
-			if (PageUptodate(p))
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
+			if (sector->uptodate)
  				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list,
-				       rbio_stripe_page(rbio, stripe, pagenr),
-				       stripe, pagenr, rbio->stripe_len,
-				       REQ_OP_READ);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 stripe, sectornr,
+						 rbio->stripe_len, REQ_OP_READ);
  			if (ret < 0)
  				goto cleanup;
  		}
@@ -2399,7 +2471,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
  	unsigned long *pbitmap = rbio->finish_pbitmap;
  	int nr_data = rbio->nr_data;
  	int stripe;
-	int pagenr;
+	int sectornr;
  	bool has_qstripe;
  	struct page *p_page = NULL;
  	struct page *q_page = NULL;
@@ -2419,7 +2491,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 
  	if (bioc->num_tgtdevs && bioc->tgtdev_map[rbio->scrubp]) {
  		is_replace = 1;
-		bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_npages);
+		bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_nsectors);
  	}
 
  	/*
@@ -2453,12 +2525,12 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
  	/* Map the parity stripe just once */
  	pointers[nr_data] = kmap_local_page(p_page);
 
-	for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
+	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
  		struct page *p;
  		void *parity;
  		/* first collect one page from each data stripe */
  		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, pagenr, 0);
+			p = page_in_rbio(rbio, stripe, sectornr, 0);
  			pointers[stripe] = kmap_local_page(p);
  		}
 
@@ -2473,13 +2545,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
  		}
 
  		/* Check scrubbing parity and repair it */
-		p = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
+		p = rbio_stripe_page(rbio, rbio->scrubp, sectornr);
  		parity = kmap_local_page(p);
  		if (memcmp(parity, pointers[rbio->scrubp], PAGE_SIZE))
  			copy_page(parity, pointers[rbio->scrubp]);
  		else
  			/* Parity is right, needn't writeback */
-			bitmap_clear(rbio->dbitmap, pagenr, 1);
+			bitmap_clear(rbio->dbitmap, sectornr, 1);
  		kunmap_local(parity);
 
  		for (stripe = nr_data - 1; stripe >= 0; stripe--)
@@ -2499,12 +2571,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
  	 * higher layers (the bio_list in our rbio) and our p/q.  Ignore
  	 * everything else.
  	 */
-	for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-		struct page *page;
+	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
+		struct sector_ptr *sector;
 
-		page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-		ret = rbio_add_io_page(rbio, &bio_list, page, rbio->scrubp,
-				       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+		sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+		ret = rbio_add_io_sector(rbio, &bio_list, sector, rbio->scrubp,
+					 sectornr, rbio->stripe_len,
+					 REQ_OP_WRITE);
  		if (ret)
  			goto cleanup;
  	}
@@ -2512,13 +2585,13 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
  	if (!is_replace)
  		goto submit_write;
 
-	for_each_set_bit(pagenr, pbitmap, rbio->stripe_npages) {
-		struct page *page;
+	for_each_set_bit(sectornr, pbitmap, rbio->stripe_nsectors) {
+		struct sector_ptr *sector;
 
-		page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-		ret = rbio_add_io_page(rbio, &bio_list, page,
+		sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+		ret = rbio_add_io_sector(rbio, &bio_list, sector,
  				       bioc->tgtdev_map[rbio->scrubp],
-				       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+				       sectornr, rbio->stripe_len, REQ_OP_WRITE);
  		if (ret)
  			goto cleanup;
  	}
@@ -2650,7 +2723,7 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)
  	int bios_to_read = 0;
  	struct bio_list bio_list;
  	int ret;
-	int pagenr;
+	int sectornr;
  	int stripe;
  	struct bio *bio;
 
@@ -2666,28 +2739,30 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)
  	 * stripe
  	 */
  	for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-		for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-			struct page *page;
+		for_each_set_bit(sectornr, rbio->dbitmap,
+				 rbio->stripe_nsectors) {
+			struct sector_ptr *sector;
  			/*
-			 * we want to find all the pages missing from
+			 * We want to find all the sectors missing from
  			 * the rbio and read them from the disk.  If
-			 * page_in_rbio finds a page in the bio list
+			 * sector_in_rbio() finds a sector in the bio list
  			 * we don't need to read it off the stripe.
  			 */
-			page = page_in_rbio(rbio, stripe, pagenr, 1);
-			if (page)
+			sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+			if (sector)
  				continue;
 
-			page = rbio_stripe_page(rbio, stripe, pagenr);
+			sector = rbio_stripe_sector(rbio, stripe, sectornr);
  			/*
-			 * the bio cache may have handed us an uptodate
-			 * page.  If so, be happy and use it
+			 * The bio cache may have handed us an uptodate
+			 * sector.  If so, be happy and use it
  			 */
-			if (PageUptodate(page))
+			if (sector->uptodate)
  				continue;
 
-			ret = rbio_add_io_page(rbio, &bio_list, page, stripe,
-					       pagenr, rbio->stripe_len, REQ_OP_READ);
+			ret = rbio_add_io_sector(rbio, &bio_list, sector,
+						 stripe, sectornr,
+						 rbio->stripe_len, REQ_OP_READ);
  			if (ret)
  				goto cleanup;
  		}
--

2.35.1

#13 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 31 lines · Qu Wenruo
Quoted 28 lines · David Sterba

There's an assertion failure reported by btrfs/023

Quoted 24 lines · Qu Wenruo

+static int rbio_add_io_sector(struct btrfs_raid_bio *rbio, +                  struct bio_list *bio_list,
+                  struct sector_ptr *sector,
+                  unsigned int stripe_nr,
+                  unsigned int sector_nr,
+                  unsigned long bio_max_len, unsigned int opf) {
+    const u32 sectorsize = rbio->bioc->fs_info->sectorsize; struct bio *last = bio_list->tail;
      int ret;
      struct bio *bio;
      struct btrfs_io_stripe *stripe;
      u64 disk_start;

+    /*
+     * NOTE: here stripe_nr has taken device replace into consideration,
+     * thus it can be larger than rbio->real_stripe.
+     * So here we check against bioc->num_stripes, not rbio->real_stripes.
+     */
+    ASSERT(stripe_nr >= 0 && stripe_nr < rbio->bioc->num_stripes); +    ASSERT(sector_nr >= 0 && sector_nr < rbio->stripe_nsectors); +    ASSERT(sector->page);

Quoted 7 lines
Quoted 1 line

This one ^^^

Failed to reproduce here, both x86_64 and aarch64 survived over 64 loops of btrfs/023.

Although that's withy my github branch. Let me check the current branch to see what's wrong.

It's a rebase error.

At least one patch has incorrect diff snippet.

For the patch "btrfs: raid56: introduce btrfs_raid_bio::stripe_sectors":

For alloc_rbio(), in my branch and patch (v2) submitted to the mailing list:

@@ -978,6 +1014,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,

        rbio = kzalloc(sizeof(*rbio) +
                       sizeof(*rbio->stripe_pages) * num_pages +
                       sizeof(*rbio->bio_pages) * num_pages +
+                      sizeof(*rbio->stripe_sectors) * num_sectors +
                       sizeof(*rbio->finish_pointers) * real_stripes +
                       sizeof(*rbio->dbitmap) * 
BITS_TO_LONGS(stripe_nsectors) +
                       sizeof(*rbio->finish_pbitmap) *
@@ -1015,6 +1052,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
        } while (0)
        CONSUME_ALLOC(rbio->stripe_pages, num_pages);
        CONSUME_ALLOC(rbio->bio_pages, num_pages);
+       CONSUME_ALLOC(rbio->stripe_sectors, num_sectors);
        CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
        CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_nsectors));
        CONSUME_ALLOC(rbio->finish_pbitmap, 

BITS_TO_LONGS(stripe_nsectors));

It has the correct extra space for stripe_sectors.

But in the for-next branch, it looks like this:
@@ -1014,6 +1050,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,

        } while (0)
        CONSUME_ALLOC(rbio->stripe_pages, num_pages);
        CONSUME_ALLOC(rbio->bio_pages, num_pages);
+       CONSUME_ALLOC(rbio->stripe_sectors, num_sectors);
        CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
        CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_nsectors));
        CONSUME_ALLOC(rbio->finish_pbitmap, 

BITS_TO_LONGS(stripe_nsectors));

Without the extra space allocated for stripe_sectors, no wonder it crashes.

Mind to provide a branch for me to rebase my patchse upon?

Thanks,
Qu

Quoted 416 lines

Thanks,
Qu

Quoted 412 lines
[ 2280.705765] assertion failed: sector->page, in fs/btrfs/raid56.c:1145
[ 2280.707844] ------------[ cut here ]------------
[ 2280.709401] kernel BUG at fs/btrfs/ctree.h:3614!
[ 2280.711084] invalid opcode: 0000 [#1] PREEMPT SMP
[ 2280.712822] CPU: 3 PID: 4084 Comm: kworker/u8:2 Not tainted 5.18.0-rc2-default+ #1697
[ 2280.715648] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014
[ 2280.719656] Workqueue: btrfs-rmw btrfs_work_helper [btrfs]
[ 2280.721775] RIP: 0010:assertfail.constprop.0+0x18/0x1a [btrfs]
[ 2280.729575] RSP: 0018:ffffad6d071afda0 EFLAGS: 00010246
[ 2280.730844] RAX: 0000000000000039 RBX: 0000000000000000 RCX: 0000000000000000
[ 2280.732449] RDX: 0000000000000000 RSI: 0000000000000003 RDI: 00000000ffffffff
[ 2280.733992] RBP: ffff8e51d5465000 R08: 0000000000000003 R09: 0000000000000002
[ 2280.735535] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000003
[ 2280.737093] R13: ffff8e51d5465000 R14: ffff8e51d5465d78 R15: 0000000000001000
[ 2280.738613] FS:  0000000000000000(0000) GS:ffff8e523dc00000(0000) knlGS:0000000000000000
[ 2280.740392] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2280.741794] CR2: 000055f48fe51ab0 CR3: 000000001f805001 CR4: 0000000000170ea0
[ 2280.743532] Call Trace:
[ 2280.744136]  <TASK>
[ 2280.744701]  rbio_add_io_sector.cold+0x11/0x33 [btrfs]
[ 2280.745846]  ? _raw_spin_unlock_irq+0x2f/0x50
[ 2280.746782]  raid56_rmw_stripe.isra.0+0x153/0x320 [btrfs]
[ 2280.747965]  btrfs_work_helper+0xd6/0x1d0 [btrfs]
[ 2280.749018]  process_one_work+0x264/0x5f0
[ 2280.749806]  worker_thread+0x52/0x3b0
[ 2280.750523]  ? process_one_work+0x5f0/0x5f0
[ 2280.751395]  kthread+0xea/0x110
[ 2280.752097]  ? kthread_complete_and_exit+0x20/0x20
[ 2280.753112]  ret_from_fork+0x1f/0x30
Quoted 380 lines
fs/btrfs/raid56.c +4 -0
+
+    /* We don't yet support subpage, thus pgoff should always be 0 */
+    ASSERT(sector->pgoff == 0);
+

      stripe = &rbio->bioc->stripes[stripe_nr];
-    disk_start = stripe->physical + (page_index << PAGE_SHIFT); +    disk_start = stripe->physical + sector_nr * sectorsize;

      /* if the device is missing, just fail this stripe */ if (!stripe->dev->bdev)

(quoted diff) +0 -0
@@ -1156,8 +1226,9 @@ static int rbio_add_io_page(struct 
 btrfs_raid_bio *rbio,
            */

          if (last_end == disk_start && !last->bi_status && last->bi_bdev == stripe->dev->bdev) {
-            ret = bio_add_page(last, page, PAGE_SIZE, 0); -            if (ret == PAGE_SIZE)
+            ret = bio_add_page(last, sector->page, sectorsize, +                       sector->pgoff);
+            if (ret == sectorsize)
                  return 0;
          }
      }

fs/btrfs/raid56.c +1 -1
@@ -1168,7 +1239,7 @@ static int rbio_add_io_page(struct 
 btrfs_raid_bio *rbio,
       bio->bi_iter.bi_sector = disk_start >> 9;
       bio->bi_private = rbio;
 
-    bio_add_page(bio, page, PAGE_SIZE, 0);
+    bio_add_page(bio, sector->page, sectorsize, sector->pgoff);
       bio_list_add(bio_list, bio);
       return 0;

  }

fs/btrfs/raid56.c +1 -1
@@ -1265,7 +1336,7 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
       void **pointers = rbio->finish_pointers;
       int nr_data = rbio->nr_data;
       int stripe;
-    int pagenr;
+    int sectornr;
       bool has_qstripe;
       struct bio_list bio_list;

      struct bio *bio;

(quoted diff) +0 -0
@@ -1309,16 +1380,16 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)

      else
          clear_bit(RBIO_CACHE_READY_BIT, &rbio->flags);

-    for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) { +    for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) { struct page *p;
          /* first collect one page from each data stripe */ for (stripe = 0; stripe < nr_data; stripe++) {
-            p = page_in_rbio(rbio, stripe, pagenr, 0);
+            p = page_in_rbio(rbio, stripe, sectornr, 0); pointers[stripe] = kmap_local_page(p);
          }

          /* then add the parity stripe */ -        p = rbio_pstripe_page(rbio, pagenr); +        p = rbio_pstripe_page(rbio, sectornr); SetPageUptodate(p); pointers[stripe++] = kmap_local_page(p);

fs/btrfs/raid56.c +9 -9
@@ -1328,7 +1399,7 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
                * raid6, add the qstripe and call the
                * library function to fill in our p/q
                */
-            p = rbio_qstripe_page(rbio, pagenr);
+            p = rbio_qstripe_page(rbio, sectornr);
               SetPageUptodate(p);
               pointers[stripe++] = kmap_local_page(p);
@@ -1349,19 +1420,19 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
        * everything else.
        */
       for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *page;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
+
               if (stripe < rbio->nr_data) {
-                page = page_in_rbio(rbio, stripe, pagenr, 1);
-                if (!page)
+                sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+                if (!sector)
                       continue;
               } else {
-                   page = rbio_stripe_page(rbio, stripe, pagenr);
+                sector = rbio_stripe_sector(rbio, stripe, sectornr);
               }
 
-            ret = rbio_add_io_page(rbio, &bio_list,
-                       page, stripe, pagenr, rbio->stripe_len,
-                       REQ_OP_WRITE);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector, stripe,
+                         sectornr, rbio->stripe_len, REQ_OP_WRITE);
               if (ret)

                  goto cleanup;
          }

fs/btrfs/raid56.c +10 -9
@@ -1374,20 +1445,21 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
           if (!bioc->tgtdev_map[stripe])
               continue;
 
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *page;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
+
               if (stripe < rbio->nr_data) {
-                page = page_in_rbio(rbio, stripe, pagenr, 1);
-                if (!page)
+                sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+                if (!sector)
                       continue;
               } else {
-                   page = rbio_stripe_page(rbio, stripe, pagenr);
+                sector = rbio_stripe_sector(rbio, stripe, sectornr);
               }
 
-            ret = rbio_add_io_page(rbio, &bio_list, page,
-                           rbio->bioc->tgtdev_map[stripe],
-                           pagenr, rbio->stripe_len,
-                           REQ_OP_WRITE);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                         rbio->bioc->tgtdev_map[stripe],
+                         sectornr, rbio->stripe_len,
+                         REQ_OP_WRITE);
               if (ret)

                  goto cleanup;
          }

fs/btrfs/raid56.c +13 -12
@@ -1563,7 +1635,7 @@ static int raid56_rmw_stripe(struct 
 btrfs_raid_bio *rbio)
       int bios_to_read = 0;
       struct bio_list bio_list;
       int ret;
-    int pagenr;
+    int sectornr;
       int stripe;
       struct bio *bio;
@@ -1581,28 +1653,29 @@ static int raid56_rmw_stripe(struct 
 btrfs_raid_bio *rbio)
        * stripe
        */
       for (stripe = 0; stripe < rbio->nr_data; stripe++) {
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *page;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
+
               /*
-             * we want to find all the pages missing from
+             * We want to find all the sectors missing from
                * the rbio and read them from the disk.  If
-             * page_in_rbio finds a page in the bio list
+             * sector_in_rbio() finds a page in the bio list
                * we don't need to read it off the stripe.
                */
-            page = page_in_rbio(rbio, stripe, pagenr, 1);
-            if (page)
+            sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+            if (sector)
                   continue;
 
-            page = rbio_stripe_page(rbio, stripe, pagenr);
+            sector = rbio_stripe_sector(rbio, stripe, sectornr);
               /*
-             * the bio cache may have handed us an uptodate
+             * The bio cache may have handed us an uptodate
                * page.  If so, be happy and use it
                */
-            if (PageUptodate(page))
+            if (sector->uptodate)
                   continue;
 
-            ret = rbio_add_io_page(rbio, &bio_list, page,
-                       stripe, pagenr, rbio->stripe_len,
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                       stripe, sectornr, rbio->stripe_len,
                          REQ_OP_READ);

              if (ret)
                  goto cleanup;

fs/btrfs/raid56.c +8 -9
@@ -2107,7 +2180,7 @@ static int __raid56_parity_recover(struct 
 btrfs_raid_bio *rbio)
       int bios_to_read = 0;
       struct bio_list bio_list;
       int ret;
-    int pagenr;
+    int sectornr;
       int stripe;
       struct bio *bio;
@@ -2130,21 +2203,20 @@ static int __raid56_parity_recover(struct 
 btrfs_raid_bio *rbio)
               continue;
           }
 
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *p;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
 
               /*
                * the rmw code may have already read this
                * page in
                */
-            p = rbio_stripe_page(rbio, stripe, pagenr);
-            if (PageUptodate(p))
+            sector = rbio_stripe_sector(rbio, stripe, sectornr);
+            if (sector->uptodate)
                   continue;
 
-            ret = rbio_add_io_page(rbio, &bio_list,
-                       rbio_stripe_page(rbio, stripe, pagenr),
-                       stripe, pagenr, rbio->stripe_len,
-                       REQ_OP_READ);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                         stripe, sectornr,
+                         rbio->stripe_len, REQ_OP_READ);
               if (ret < 0)

                  goto cleanup;
          }

fs/btrfs/raid56.c +1 -1
@@ -2399,7 +2471,7 @@ static noinline void finish_parity_scrub(struct 
 btrfs_raid_bio *rbio,
       unsigned long *pbitmap = rbio->finish_pbitmap;
       int nr_data = rbio->nr_data;
       int stripe;
-    int pagenr;
+    int sectornr;
       bool has_qstripe;
       struct page *p_page = NULL;

      struct page *q_page = NULL;

fs/btrfs/raid56.c +1 -1
@@ -2419,7 +2491,7 @@ static noinline void finish_parity_scrub(struct 
 btrfs_raid_bio *rbio,
 
       if (bioc->num_tgtdevs && bioc->tgtdev_map[rbio->scrubp]) {
           is_replace = 1;
-        bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_npages);
+        bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_nsectors);
       }
 

      /*

fs/btrfs/raid56.c +3 -3
@@ -2453,12 +2525,12 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
       /* Map the parity stripe just once */
       pointers[nr_data] = kmap_local_page(p_page);
 
-    for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
+    for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
           struct page *p;
           void *parity;
           /* first collect one page from each data stripe */
           for (stripe = 0; stripe < nr_data; stripe++) {
-            p = page_in_rbio(rbio, stripe, pagenr, 0);
+            p = page_in_rbio(rbio, stripe, sectornr, 0);
               pointers[stripe] = kmap_local_page(p);
           }
@@ -2473,13 +2545,13 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
           }
 
           /* Check scrubbing parity and repair it */
-        p = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
+        p = rbio_stripe_page(rbio, rbio->scrubp, sectornr);
           parity = kmap_local_page(p);
           if (memcmp(parity, pointers[rbio->scrubp], PAGE_SIZE))
               copy_page(parity, pointers[rbio->scrubp]);

          else
              /* Parity is right, needn't writeback */ -            bitmap_clear(rbio->dbitmap, pagenr, 1); +            bitmap_clear(rbio->dbitmap, sectornr, 1); kunmap_local(parity);

          for (stripe = nr_data - 1; stripe >= 0; stripe--)

fs/btrfs/raid56.c +6 -5
@@ -2499,12 +2571,13 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
        * higher layers (the bio_list in our rbio) and our p/q.  Ignore
        * everything else.
        */
-    for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-        struct page *page;
+    for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
+        struct sector_ptr *sector;
 
-        page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-        ret = rbio_add_io_page(rbio, &bio_list, page, rbio->scrubp,
-                       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+        sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+        ret = rbio_add_io_sector(rbio, &bio_list, sector, rbio->scrubp,
+                     sectornr, rbio->stripe_len,
+                     REQ_OP_WRITE);
           if (ret)
               goto cleanup;

      }

fs/btrfs/raid56.c +5 -5
@@ -2512,13 +2585,13 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
       if (!is_replace)
           goto submit_write;
 
-    for_each_set_bit(pagenr, pbitmap, rbio->stripe_npages) {
-        struct page *page;
+    for_each_set_bit(sectornr, pbitmap, rbio->stripe_nsectors) {
+        struct sector_ptr *sector;
 
-        page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-        ret = rbio_add_io_page(rbio, &bio_list, page,
+        sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+        ret = rbio_add_io_sector(rbio, &bio_list, sector,
                          bioc->tgtdev_map[rbio->scrubp],
-                       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+                       sectornr, rbio->stripe_len, REQ_OP_WRITE);
           if (ret)
               goto cleanup;

      }

fs/btrfs/raid56.c +15 -13
@@ -2650,7 +2723,7 @@ static void raid56_parity_scrub_stripe(struct 
 btrfs_raid_bio *rbio)
       int bios_to_read = 0;
       struct bio_list bio_list;
       int ret;
-    int pagenr;
+    int sectornr;
       int stripe;
       struct bio *bio;
@@ -2666,28 +2739,30 @@ static void raid56_parity_scrub_stripe(struct 
 btrfs_raid_bio *rbio)
        * stripe
        */
       for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-        for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-            struct page *page;
+        for_each_set_bit(sectornr, rbio->dbitmap,
+                 rbio->stripe_nsectors) {
+            struct sector_ptr *sector;
               /*
-             * we want to find all the pages missing from
+             * We want to find all the sectors missing from
                * the rbio and read them from the disk.  If
-             * page_in_rbio finds a page in the bio list
+             * sector_in_rbio() finds a sector in the bio list
                * we don't need to read it off the stripe.
                */
-            page = page_in_rbio(rbio, stripe, pagenr, 1);
-            if (page)
+            sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+            if (sector)
                   continue;
 
-            page = rbio_stripe_page(rbio, stripe, pagenr);
+            sector = rbio_stripe_sector(rbio, stripe, sectornr);
               /*
-             * the bio cache may have handed us an uptodate
-             * page.  If so, be happy and use it
+             * The bio cache may have handed us an uptodate
+             * sector.  If so, be happy and use it
                */
-            if (PageUptodate(page))
+            if (sector->uptodate)
                   continue;
 
-            ret = rbio_add_io_page(rbio, &bio_list, page, stripe,
-                           pagenr, rbio->stripe_len, REQ_OP_READ);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                         stripe, sectornr,
+                         rbio->stripe_len, REQ_OP_READ);
               if (ret)
                   goto cleanup;

          }

Signature · 1 line
2.35.1
#14 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 51 lines · Qu Wenruo
Quoted 31 lines · Qu Wenruo
Quoted 28 lines · David Sterba

There's an assertion failure reported by btrfs/023

Quoted 24 lines · Qu Wenruo

+static int rbio_add_io_sector(struct btrfs_raid_bio *rbio, +                  struct bio_list *bio_list,
+                  struct sector_ptr *sector,
+                  unsigned int stripe_nr,
+                  unsigned int sector_nr,
+                  unsigned long bio_max_len, unsigned int opf) {
+    const u32 sectorsize = rbio->bioc->fs_info->sectorsize; struct bio *last = bio_list->tail;
      int ret;
      struct bio *bio;
      struct btrfs_io_stripe *stripe;
      u64 disk_start;

+    /*
+     * NOTE: here stripe_nr has taken device replace into consideration,
+     * thus it can be larger than rbio->real_stripe.
+     * So here we check against bioc->num_stripes, not rbio->real_stripes.
+     */
+    ASSERT(stripe_nr >= 0 && stripe_nr < rbio->bioc->num_stripes); +    ASSERT(sector_nr >= 0 && sector_nr < rbio->stripe_nsectors); +    ASSERT(sector->page);

Quoted 7 lines
Quoted 1 line

This one ^^^

Failed to reproduce here, both x86_64 and aarch64 survived over 64 loops of btrfs/023.

Although that's withy my github branch. Let me check the current branch to see what's wrong.

It's a rebase error.

At least one patch has incorrect diff snippet.

For the patch "btrfs: raid56: introduce btrfs_raid_bio::stripe_sectors":

For alloc_rbio(), in my branch and patch (v2) submitted to the mailing list:

Review static struct btrfs_raid_bio *alloc_rbio(struct · line 1018

Quoted 5 lines
(quoted diff) +1 -0
@@ -978,6 +1014,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct 
 btrfs_fs_info *fs_info,
         rbio = kzalloc(sizeof(*rbio) +
                        sizeof(*rbio->stripe_pages) * num_pages +
                        sizeof(*rbio->bio_pages) * num_pages +
+                      sizeof(*rbio->stripe_sectors) * num_sectors +

The more I check this part of the existing code, the worse it looks to me.

We're really dancing on the edge, it's just a problem of time for us to get a cut.

Shouldn't we go the regular way, kzalloc the structure only, then alloc needed pages/bitmaps?

Thanks,
Qu

Quoted 453 lines
(quoted diff) +0 -0
                        sizeof(*rbio->finish_pointers) * real_stripes +
                        sizeof(*rbio->dbitmap) * 

BITS_TO_LONGS(stripe_nsectors) +
                       sizeof(*rbio->finish_pbitmap) *

(quoted diff) +1 -0
@@ -1015,6 +1052,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct 
 btrfs_fs_info *fs_info,
         } while (0)
         CONSUME_ALLOC(rbio->stripe_pages, num_pages);
         CONSUME_ALLOC(rbio->bio_pages, num_pages);
+       CONSUME_ALLOC(rbio->stripe_sectors, num_sectors);
         CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
         CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_nsectors));

        CONSUME_ALLOC(rbio->finish_pbitmap,
BITS_TO_LONGS(stripe_nsectors));

It has the correct extra space for stripe_sectors.

But in the for-next branch, it looks like this:

(quoted diff) +1 -0
@@ -1014,6 +1050,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct 
 btrfs_fs_info *fs_info,
         } while (0)
         CONSUME_ALLOC(rbio->stripe_pages, num_pages);
         CONSUME_ALLOC(rbio->bio_pages, num_pages);
+       CONSUME_ALLOC(rbio->stripe_sectors, num_sectors);
         CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
         CONSUME_ALLOC(rbio->dbitmap, BITS_TO_LONGS(stripe_nsectors));

        CONSUME_ALLOC(rbio->finish_pbitmap,
BITS_TO_LONGS(stripe_nsectors));

Without the extra space allocated for stripe_sectors, no wonder it crashes.

Mind to provide a branch for me to rebase my patchse upon?

Thanks,
Qu

Quoted 417 lines

Thanks,
Qu

Quoted 413 lines
[ 2280.705765] assertion failed: sector->page, in fs/btrfs/raid56.c:1145
[ 2280.707844] ------------[ cut here ]------------
[ 2280.709401] kernel BUG at fs/btrfs/ctree.h:3614!
[ 2280.711084] invalid opcode: 0000 [#1] PREEMPT SMP
[ 2280.712822] CPU: 3 PID: 4084 Comm: kworker/u8:2 Not tainted 5.18.0-rc2-default+ #1697
[ 2280.715648] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014
[ 2280.719656] Workqueue: btrfs-rmw btrfs_work_helper [btrfs]
[ 2280.721775] RIP: 0010:assertfail.constprop.0+0x18/0x1a [btrfs]
[ 2280.729575] RSP: 0018:ffffad6d071afda0 EFLAGS: 00010246
[ 2280.730844] RAX: 0000000000000039 RBX: 0000000000000000 RCX: 0000000000000000
[ 2280.732449] RDX: 0000000000000000 RSI: 0000000000000003 RDI: 00000000ffffffff
[ 2280.733992] RBP: ffff8e51d5465000 R08: 0000000000000003 R09: 0000000000000002
[ 2280.735535] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000003
[ 2280.737093] R13: ffff8e51d5465000 R14: ffff8e51d5465d78 R15: 0000000000001000
[ 2280.738613] FS:  0000000000000000(0000) GS:ffff8e523dc00000(0000) knlGS:0000000000000000
[ 2280.740392] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2280.741794] CR2: 000055f48fe51ab0 CR3: 000000001f805001 CR4: 0000000000170ea0
[ 2280.743532] Call Trace:
[ 2280.744136]  <TASK>
[ 2280.744701]  rbio_add_io_sector.cold+0x11/0x33 [btrfs]
[ 2280.745846]  ? _raw_spin_unlock_irq+0x2f/0x50
[ 2280.746782]  raid56_rmw_stripe.isra.0+0x153/0x320 [btrfs]
[ 2280.747965]  btrfs_work_helper+0xd6/0x1d0 [btrfs]
[ 2280.749018]  process_one_work+0x264/0x5f0
[ 2280.749806]  worker_thread+0x52/0x3b0
[ 2280.750523]  ? process_one_work+0x5f0/0x5f0
[ 2280.751395]  kthread+0xea/0x110
[ 2280.752097]  ? kthread_complete_and_exit+0x20/0x20
[ 2280.753112]  ret_from_fork+0x1f/0x30
Quoted 381 lines
fs/btrfs/raid56.c +4 -0
+
+    /* We don't yet support subpage, thus pgoff should always be 0 */
+    ASSERT(sector->pgoff == 0);
+

      stripe = &rbio->bioc->stripes[stripe_nr];
-    disk_start = stripe->physical + (page_index << PAGE_SHIFT); +    disk_start = stripe->physical + sector_nr * sectorsize;

      /* if the device is missing, just fail this stripe */ if (!stripe->dev->bdev)

(quoted diff) +0 -0
@@ -1156,8 +1226,9 @@ static int rbio_add_io_page(struct 
 btrfs_raid_bio *rbio,
            */

          if (last_end == disk_start && !last->bi_status && last->bi_bdev == stripe->dev->bdev) {
-            ret = bio_add_page(last, page, PAGE_SIZE, 0); -            if (ret == PAGE_SIZE)
+            ret = bio_add_page(last, sector->page, sectorsize, +                       sector->pgoff);
+            if (ret == sectorsize)
                  return 0;
          }
      }

fs/btrfs/raid56.c +1 -1
@@ -1168,7 +1239,7 @@ static int rbio_add_io_page(struct 
 btrfs_raid_bio *rbio,
       bio->bi_iter.bi_sector = disk_start >> 9;
       bio->bi_private = rbio;
 
-    bio_add_page(bio, page, PAGE_SIZE, 0);
+    bio_add_page(bio, sector->page, sectorsize, sector->pgoff);
       bio_list_add(bio_list, bio);
       return 0;

  }

fs/btrfs/raid56.c +1 -1
@@ -1265,7 +1336,7 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
       void **pointers = rbio->finish_pointers;
       int nr_data = rbio->nr_data;
       int stripe;
-    int pagenr;
+    int sectornr;
       bool has_qstripe;
       struct bio_list bio_list;

      struct bio *bio;

(quoted diff) +0 -0
@@ -1309,16 +1380,16 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)

      else
          clear_bit(RBIO_CACHE_READY_BIT, &rbio->flags);

-    for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) { +    for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) { struct page *p;
          /* first collect one page from each data stripe */ for (stripe = 0; stripe < nr_data; stripe++) {
-            p = page_in_rbio(rbio, stripe, pagenr, 0);
+            p = page_in_rbio(rbio, stripe, sectornr, 0); pointers[stripe] = kmap_local_page(p);
          }

          /* then add the parity stripe */ -        p = rbio_pstripe_page(rbio, pagenr); +        p = rbio_pstripe_page(rbio, sectornr); SetPageUptodate(p); pointers[stripe++] = kmap_local_page(p);

fs/btrfs/raid56.c +9 -9
@@ -1328,7 +1399,7 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
                * raid6, add the qstripe and call the
                * library function to fill in our p/q
                */
-            p = rbio_qstripe_page(rbio, pagenr);
+            p = rbio_qstripe_page(rbio, sectornr);
               SetPageUptodate(p);
               pointers[stripe++] = kmap_local_page(p);
@@ -1349,19 +1420,19 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
        * everything else.
        */
       for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *page;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
+
               if (stripe < rbio->nr_data) {
-                page = page_in_rbio(rbio, stripe, pagenr, 1);
-                if (!page)
+                sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+                if (!sector)
                       continue;
               } else {
-                   page = rbio_stripe_page(rbio, stripe, pagenr);
+                sector = rbio_stripe_sector(rbio, stripe, sectornr);
               }
 
-            ret = rbio_add_io_page(rbio, &bio_list,
-                       page, stripe, pagenr, rbio->stripe_len,
-                       REQ_OP_WRITE);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector, stripe,
+                         sectornr, rbio->stripe_len, REQ_OP_WRITE);
               if (ret)

                  goto cleanup;
          }

fs/btrfs/raid56.c +10 -9
@@ -1374,20 +1445,21 @@ static noinline void finish_rmw(struct 
 btrfs_raid_bio *rbio)
           if (!bioc->tgtdev_map[stripe])
               continue;
 
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *page;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
+
               if (stripe < rbio->nr_data) {
-                page = page_in_rbio(rbio, stripe, pagenr, 1);
-                if (!page)
+                sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+                if (!sector)
                       continue;
               } else {
-                   page = rbio_stripe_page(rbio, stripe, pagenr);
+                sector = rbio_stripe_sector(rbio, stripe, sectornr);
               }
 
-            ret = rbio_add_io_page(rbio, &bio_list, page,
-                           rbio->bioc->tgtdev_map[stripe],
-                           pagenr, rbio->stripe_len,
-                           REQ_OP_WRITE);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                         rbio->bioc->tgtdev_map[stripe],
+                         sectornr, rbio->stripe_len,
+                         REQ_OP_WRITE);
               if (ret)

                  goto cleanup;
          }

fs/btrfs/raid56.c +13 -12
@@ -1563,7 +1635,7 @@ static int raid56_rmw_stripe(struct 
 btrfs_raid_bio *rbio)
       int bios_to_read = 0;
       struct bio_list bio_list;
       int ret;
-    int pagenr;
+    int sectornr;
       int stripe;
       struct bio *bio;
@@ -1581,28 +1653,29 @@ static int raid56_rmw_stripe(struct 
 btrfs_raid_bio *rbio)
        * stripe
        */
       for (stripe = 0; stripe < rbio->nr_data; stripe++) {
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *page;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
+
               /*
-             * we want to find all the pages missing from
+             * We want to find all the sectors missing from
                * the rbio and read them from the disk.  If
-             * page_in_rbio finds a page in the bio list
+             * sector_in_rbio() finds a page in the bio list
                * we don't need to read it off the stripe.
                */
-            page = page_in_rbio(rbio, stripe, pagenr, 1);
-            if (page)
+            sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+            if (sector)
                   continue;
 
-            page = rbio_stripe_page(rbio, stripe, pagenr);
+            sector = rbio_stripe_sector(rbio, stripe, sectornr);
               /*
-             * the bio cache may have handed us an uptodate
+             * The bio cache may have handed us an uptodate
                * page.  If so, be happy and use it
                */
-            if (PageUptodate(page))
+            if (sector->uptodate)
                   continue;
 
-            ret = rbio_add_io_page(rbio, &bio_list, page,
-                       stripe, pagenr, rbio->stripe_len,
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                       stripe, sectornr, rbio->stripe_len,
                          REQ_OP_READ);

              if (ret)
                  goto cleanup;

fs/btrfs/raid56.c +8 -9
@@ -2107,7 +2180,7 @@ static int __raid56_parity_recover(struct 
 btrfs_raid_bio *rbio)
       int bios_to_read = 0;
       struct bio_list bio_list;
       int ret;
-    int pagenr;
+    int sectornr;
       int stripe;
       struct bio *bio;
@@ -2130,21 +2203,20 @@ static int __raid56_parity_recover(struct 
 btrfs_raid_bio *rbio)
               continue;
           }
 
-        for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
-            struct page *p;
+        for (sectornr = 0; sectornr < rbio->stripe_nsectors; 
 sectornr++) {
+            struct sector_ptr *sector;
 
               /*
                * the rmw code may have already read this
                * page in
                */
-            p = rbio_stripe_page(rbio, stripe, pagenr);
-            if (PageUptodate(p))
+            sector = rbio_stripe_sector(rbio, stripe, sectornr);
+            if (sector->uptodate)
                   continue;
 
-            ret = rbio_add_io_page(rbio, &bio_list,
-                       rbio_stripe_page(rbio, stripe, pagenr),
-                       stripe, pagenr, rbio->stripe_len,
-                       REQ_OP_READ);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                         stripe, sectornr,
+                         rbio->stripe_len, REQ_OP_READ);
               if (ret < 0)

                  goto cleanup;
          }

fs/btrfs/raid56.c +1 -1
@@ -2399,7 +2471,7 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
       unsigned long *pbitmap = rbio->finish_pbitmap;
       int nr_data = rbio->nr_data;
       int stripe;
-    int pagenr;
+    int sectornr;
       bool has_qstripe;
       struct page *p_page = NULL;

      struct page *q_page = NULL;

fs/btrfs/raid56.c +1 -1
@@ -2419,7 +2491,7 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
 
       if (bioc->num_tgtdevs && bioc->tgtdev_map[rbio->scrubp]) {
           is_replace = 1;
-        bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_npages);
+        bitmap_copy(pbitmap, rbio->dbitmap, rbio->stripe_nsectors);
       }
 

      /*

fs/btrfs/raid56.c +3 -3
@@ -2453,12 +2525,12 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
       /* Map the parity stripe just once */
       pointers[nr_data] = kmap_local_page(p_page);
 
-    for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
+    for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
           struct page *p;
           void *parity;
           /* first collect one page from each data stripe */
           for (stripe = 0; stripe < nr_data; stripe++) {
-            p = page_in_rbio(rbio, stripe, pagenr, 0);
+            p = page_in_rbio(rbio, stripe, sectornr, 0);
               pointers[stripe] = kmap_local_page(p);
           }
@@ -2473,13 +2545,13 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
           }
 
           /* Check scrubbing parity and repair it */
-        p = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
+        p = rbio_stripe_page(rbio, rbio->scrubp, sectornr);
           parity = kmap_local_page(p);
           if (memcmp(parity, pointers[rbio->scrubp], PAGE_SIZE))
               copy_page(parity, pointers[rbio->scrubp]);

          else
              /* Parity is right, needn't writeback */ -            bitmap_clear(rbio->dbitmap, pagenr, 1); +            bitmap_clear(rbio->dbitmap, sectornr, 1); kunmap_local(parity);

          for (stripe = nr_data - 1; stripe >= 0; stripe--)

fs/btrfs/raid56.c +6 -5
@@ -2499,12 +2571,13 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
        * higher layers (the bio_list in our rbio) and our p/q.  Ignore
        * everything else.
        */
-    for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-        struct page *page;
+    for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
+        struct sector_ptr *sector;
 
-        page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-        ret = rbio_add_io_page(rbio, &bio_list, page, rbio->scrubp,
-                       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+        sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+        ret = rbio_add_io_sector(rbio, &bio_list, sector, 
 rbio->scrubp,
+                     sectornr, rbio->stripe_len,
+                     REQ_OP_WRITE);
           if (ret)

              goto cleanup;
      }

fs/btrfs/raid56.c +5 -5
@@ -2512,13 +2585,13 @@ static noinline void 
 finish_parity_scrub(struct btrfs_raid_bio *rbio,
       if (!is_replace)
           goto submit_write;
 
-    for_each_set_bit(pagenr, pbitmap, rbio->stripe_npages) {
-        struct page *page;
+    for_each_set_bit(sectornr, pbitmap, rbio->stripe_nsectors) {
+        struct sector_ptr *sector;
 
-        page = rbio_stripe_page(rbio, rbio->scrubp, pagenr);
-        ret = rbio_add_io_page(rbio, &bio_list, page,
+        sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+        ret = rbio_add_io_sector(rbio, &bio_list, sector,
                          bioc->tgtdev_map[rbio->scrubp],
-                       pagenr, rbio->stripe_len, REQ_OP_WRITE);
+                       sectornr, rbio->stripe_len, REQ_OP_WRITE);
           if (ret)
               goto cleanup;

      }

fs/btrfs/raid56.c +15 -13
@@ -2650,7 +2723,7 @@ static void raid56_parity_scrub_stripe(struct 
 btrfs_raid_bio *rbio)
       int bios_to_read = 0;
       struct bio_list bio_list;
       int ret;
-    int pagenr;
+    int sectornr;
       int stripe;
       struct bio *bio;
@@ -2666,28 +2739,30 @@ static void 
 raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)
        * stripe
        */
       for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
-        for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
-            struct page *page;
+        for_each_set_bit(sectornr, rbio->dbitmap,
+                 rbio->stripe_nsectors) {
+            struct sector_ptr *sector;
               /*
-             * we want to find all the pages missing from
+             * We want to find all the sectors missing from
                * the rbio and read them from the disk.  If
-             * page_in_rbio finds a page in the bio list
+             * sector_in_rbio() finds a sector in the bio list
                * we don't need to read it off the stripe.
                */
-            page = page_in_rbio(rbio, stripe, pagenr, 1);
-            if (page)
+            sector = sector_in_rbio(rbio, stripe, sectornr, 1);
+            if (sector)
                   continue;
 
-            page = rbio_stripe_page(rbio, stripe, pagenr);
+            sector = rbio_stripe_sector(rbio, stripe, sectornr);
               /*
-             * the bio cache may have handed us an uptodate
-             * page.  If so, be happy and use it
+             * The bio cache may have handed us an uptodate
+             * sector.  If so, be happy and use it
                */
-            if (PageUptodate(page))
+            if (sector->uptodate)
                   continue;
 
-            ret = rbio_add_io_page(rbio, &bio_list, page, stripe,
-                           pagenr, rbio->stripe_len, REQ_OP_READ);
+            ret = rbio_add_io_sector(rbio, &bio_list, sector,
+                         stripe, sectornr,
+                         rbio->stripe_len, REQ_OP_READ);
               if (ret)
                   goto cleanup;

          }

Signature · 1 line
2.35.1
#15 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 20 lines · Qu Wenruo
Quoted 12 lines
Quoted 1 line

to see what's wrong.

It's a rebase error.

At least one patch has incorrect diff snippet.

For the patch "btrfs: raid56: introduce btrfs_raid_bio::stripe_sectors":

For alloc_rbio(), in my branch and patch (v2) submitted to the mailing list:

Review static struct btrfs_raid_bio *alloc_rbio(struct · line 1018

Quoted 5 lines
(quoted diff) +1 -0
@@ -978,6 +1014,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct 
 btrfs_fs_info *fs_info,
         rbio = kzalloc(sizeof(*rbio) +
                        sizeof(*rbio->stripe_pages) * num_pages +
                        sizeof(*rbio->bio_pages) * num_pages +
+                      sizeof(*rbio->stripe_sectors) * num_sectors +

The more I check this part of the existing code, the worse it looks to me.

Quoted 5 lines

We're really dancing on the edge, it's just a problem of time for us to get a cut.

Shouldn't we go the regular way, kzalloc the structure only, then alloc needed pages/bitmaps?

Yeah I agree that this has become unwieldy with the definitions and calculations. One thing to consider is the real memory footprint and the overhead of one kmalloc vs several kmalloc of smaller memory chunks. Both have pros and cons but if there is no real effect then we may have better code with the separate allocations.

#16 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 24 lines · David Sterba
Quoted 23 lines · Qu Wenruo
Quoted 12 lines
Quoted 1 line

to see what's wrong.

It's a rebase error.

At least one patch has incorrect diff snippet.

For the patch "btrfs: raid56: introduce btrfs_raid_bio::stripe_sectors":

For alloc_rbio(), in my branch and patch (v2) submitted to the mailing list:

Review static struct btrfs_raid_bio *alloc_rbio(struct · line 1018

Quoted 5 lines
(quoted diff) +1 -0
@@ -978,6 +1014,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct
 btrfs_fs_info *fs_info,
          rbio = kzalloc(sizeof(*rbio) +
                         sizeof(*rbio->stripe_pages) * num_pages +
                         sizeof(*rbio->bio_pages) * num_pages +
+                      sizeof(*rbio->stripe_sectors) * num_sectors +

The more I check this part of the existing code, the worse it looks to me.

We're really dancing on the edge, it's just a problem of time for us to get a cut.

Quoted 6 lines
Quoted 2 lines

Shouldn't we go the regular way, kzalloc the structure only, then alloc needed pages/bitmaps?

Yeah I agree that this has become unwieldy with the definitions and calculations. One thing to consider is the real memory footprint and the overhead of one kmalloc vs several kmalloc of smaller memory chunks.

One thing to mention is, although our one kmalloc seems to be some optimization, I guess the benefit is already small or none, especially after subpage patchset.

Just check for 3 disks RAID5, one btrfs_raid_bio already takes over 2Kilo:

After: 2320 (+97.8%) (From the cover letter)

In this case, I doubt smaller kmallocs would cause anything worse, as kmalloc() will give us a 4K page/range anyway.

Quoted 2 lines

Both have pros and cons but if there is no real effect then we may have better code with the separate allocations.

There are some ways to optimize, but they may be a little aggressive.

The most aggressive one would be, replace "struct page **stripe_pages" to "struct page *stripe_page", and use alloc_pages_exact() to allocate a range of physically adjacent pages.

In fact I'm even considering this for extent buffer pages, but it would cause way more unexpected ENOMEM when memory is fragmented, thus too aggressive.

Thanks,
Qu

#17 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 38 lines · Qu Wenruo
Quoted 24 lines · David Sterba
Quoted 23 lines · Qu Wenruo
Quoted 12 lines
Quoted 1 line

to see what's wrong.

It's a rebase error.

At least one patch has incorrect diff snippet.

For the patch "btrfs: raid56: introduce btrfs_raid_bio::stripe_sectors":

For alloc_rbio(), in my branch and patch (v2) submitted to the mailing list:

Review static struct btrfs_raid_bio *alloc_rbio(struct · line 1018

Quoted 5 lines
(quoted diff) +1 -0
@@ -978,6 +1014,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct
 btrfs_fs_info *fs_info,
          rbio = kzalloc(sizeof(*rbio) +
                         sizeof(*rbio->stripe_pages) * num_pages +
                         sizeof(*rbio->bio_pages) * num_pages +
+                      sizeof(*rbio->stripe_sectors) * num_sectors +

The more I check this part of the existing code, the worse it looks to me.

We're really dancing on the edge, it's just a problem of time for us to get a cut.

Quoted 6 lines
Quoted 2 lines

Shouldn't we go the regular way, kzalloc the structure only, then alloc needed pages/bitmaps?

Yeah I agree that this has become unwieldy with the definitions and calculations. One thing to consider is the real memory footprint and the overhead of one kmalloc vs several kmalloc of smaller memory chunks.

One thing to mention is, although our one kmalloc seems to be some optimization, I guess the benefit is already small or none, especially after subpage patchset.

Quoted 6 lines

Just check for 3 disks RAID5, one btrfs_raid_bio already takes over 2Kilo:

After: 2320 (+97.8%) (From the cover letter)

In this case, I doubt smaller kmallocs would cause anything worse, as kmalloc() will give us a 4K page/range anyway.

Small allocations fall into the kmalloc-<size> slabs.

Quoted 5 lines
Quoted 2 lines

Both have pros and cons but if there is no real effect then we may have better code with the separate allocations.

There are some ways to optimize, but they may be a little aggressive.

Quoted 7 lines

The most aggressive one would be, replace "struct page **stripe_pages" to "struct page *stripe_page", and use alloc_pages_exact() to allocate a range of physically adjacent pages.

In fact I'm even considering this for extent buffer pages, but it would cause way more unexpected ENOMEM when memory is fragmented, thus too aggressive.

Yeah we must assume the memory can get fragmented so allocating individual pages must stay as a fallback. As extent bufferes represent the metadata, we can't afford to return such errors in case there's enough memory but not in the shape we'd like it.

I've thought about using the contig allocaiton too as it makes the memcpy in the eb helpers much easier. What could be done is to have both, optimistically allocating contiguous range and falling back to what we have now, tracking the status in the eb flags. This means that there would be a fast path if we're lucky and get the contiguous range and this could speed some things up.

#18 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

In reply to Qu Wenruo
Quoted 9 lines · Qu Wenruo
Quoted 5 lines

Failed to reproduce here, both x86_64 and aarch64 survived over 64 loops of btrfs/023.

Although that's withy my github branch. Let me check the current branch to see what's wrong.

It's a rebase error.

At least one patch has incorrect diff snippet.

Ok, my bad, I did not do a finall diff check, I'll update the patchset.

Quoted 1 line

Mind to provide a branch for me to rebase my patchse upon?

It was is for-next I guess you found it already.

#19 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 4 lines · David Sterba
Quoted 3 lines · Qu Wenruo
Quoted 3 lines

Failed to reproduce here, both x86_64 and aarch64 survived over 64 loops of btrfs/023.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Although that's withy my github branch. Let me check the current branch to see what's wrong.

It's a rebase error.

At least one patch has incorrect diff snippet.

Ok, my bad, I did not do a finall diff check, I'll update the patchset.

I should have done that, there are way more code changes that I missed when going through the patch difference manually. I'll try to port it again but uh.

#20 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 10 lines · David Sterba
Quoted 9 lines · David Sterba
Quoted 8 lines · Qu Wenruo
Quoted 5 lines

Failed to reproduce here, both x86_64 and aarch64 survived over 64 loops of btrfs/023.

Although that's withy my github branch. Let me check the current branch to see what's wrong.

It's a rebase error.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

At least one patch has incorrect diff snippet.

Ok, my bad, I did not do a finall diff check, I'll update the patchset.

I should have done that, there are way more code changes that I missed when going through the patch difference manually. I'll try to port it again but uh.

Mind me to do the rebase?

Just base all the changes to the last commit just before the branch.

As doing manually diff check would also give me a deeper impression on what I should do in the first place to reduce your workload.

THanks,
Qu

#21 lore raw

Re: [PATCH v2 07/17] btrfs: make rbio_add_io_page() subpage compatible

Quoted 10 lines · Qu Wenruo
Quoted 3 lines

I should have done that, there are way more code changes that I missed when going through the patch difference manually. I'll try to port it again but uh.

Mind me to do the rebase?

Just base all the changes to the last commit just before the branch.

As doing manually diff check would also give me a deeper impression on what I should do in the first place to reduce your workload.

I don't push all the topic branches but they appear in for-next so you can extract it from there, otherwise I can always push it on request so we can sort it out.

Anyway, the branch is now merged to misc-next, and regarding the problems I think there were a few factors that struck at the same time. Checking diff of my changes against your branch is a safety check that I should have done. We want to get branches for testing but patch revisions also happen, so they can diverge and that we all stick to the same coding style and formatting preferences is a noble goal but practicaly not 100% achievable, there's always something left.

author
#22 lore raw

PATCH v2 8/17 btrfs: make finish_parity_scrub() subpage compatible

In reply to Qu Wenruo

The core is to convert direct page usage into sector_ptr usage, and use memcpy() to replace copy_page().

For pointers usage, we need to convert it to kmap_local_page() + sector->pgoff.

---
 fs/btrfs/raid56.c | 56 +++++++++++++++++++++++++++--------------------
 1 file changed, 32 insertions(+), 24 deletions(-)
fs/btrfs/raid56.c +32 -24
index 9c3e0201d032..6b2d75a09896 100644
@@ -2467,14 +2467,15 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 					 int need_check)
 {
 	struct btrfs_io_context *bioc = rbio->bioc;
+	const u32 sectorsize = bioc->fs_info->sectorsize;
 	void **pointers = rbio->finish_pointers;
 	unsigned long *pbitmap = rbio->finish_pbitmap;
 	int nr_data = rbio->nr_data;
 	int stripe;
 	int sectornr;
 	bool has_qstripe;
-	struct page *p_page = NULL;
-	struct page *q_page = NULL;
+	struct sector_ptr p_sector = { 0 };
+	struct sector_ptr q_sector = { 0 };
 	struct bio_list bio_list;
 	struct bio *bio;
 	int is_replace = 0;
@@ -2504,51 +2505,56 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	if (!need_check)
 		goto writeback;
 
-	p_page = alloc_page(GFP_NOFS);
-	if (!p_page)
+	p_sector.page = alloc_page(GFP_NOFS);
+	if (!p_sector.page)
 		goto cleanup;
-	SetPageUptodate(p_page);
+	p_sector.pgoff = 0;
+	p_sector.uptodate = 1;
 
 	if (has_qstripe) {
 		/* RAID6, allocate and map temp space for the Q stripe */
-		q_page = alloc_page(GFP_NOFS);
-		if (!q_page) {
-			__free_page(p_page);
+		q_sector.page = alloc_page(GFP_NOFS);
+		if (!q_sector.page) {
+			__free_page(p_sector.page);
+			p_sector.page = NULL;
 			goto cleanup;
 		}
-		SetPageUptodate(q_page);
-		pointers[rbio->real_stripes - 1] = kmap_local_page(q_page);
+		q_sector.pgoff = 0;
+		q_sector.uptodate = 1;
+		pointers[rbio->real_stripes - 1] = kmap_local_page(q_sector.page);
 	}
 
 	atomic_set(&rbio->error, 0);
 
 	/* Map the parity stripe just once */
-	pointers[nr_data] = kmap_local_page(p_page);
+	pointers[nr_data] = kmap_local_page(p_sector.page);
 
 	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
-		struct page *p;
+		struct sector_ptr *sector;
 		void *parity;
+
 		/* first collect one page from each data stripe */
 		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, sectornr, 0);
-			pointers[stripe] = kmap_local_page(p);
+			sector = sector_in_rbio(rbio, stripe, sectornr, 0);
+			pointers[stripe] = kmap_local_page(sector->page) +
+					   sector->pgoff;
 		}
 
 		if (has_qstripe) {
 			/* RAID6, call the library function to fill in our P/Q */
-			raid6_call.gen_syndrome(rbio->real_stripes, PAGE_SIZE,
+			raid6_call.gen_syndrome(rbio->real_stripes, sectorsize,
 						pointers);
 		} else {
 			/* raid5 */
-			copy_page(pointers[nr_data], pointers[0]);
-			run_xor(pointers + 1, nr_data - 1, PAGE_SIZE);
+			memcpy(pointers[nr_data], pointers[0], sectorsize);
+			run_xor(pointers + 1, nr_data - 1, sectorsize);
 		}
 
 		/* Check scrubbing parity and repair it */
-		p = rbio_stripe_page(rbio, rbio->scrubp, sectornr);
-		parity = kmap_local_page(p);
-		if (memcmp(parity, pointers[rbio->scrubp], PAGE_SIZE))
-			copy_page(parity, pointers[rbio->scrubp]);
+		sector = rbio_stripe_sector(rbio, rbio->scrubp, sectornr);
+		parity = kmap_local_page(sector->page) + sector->pgoff;
+		if (memcmp(parity, pointers[rbio->scrubp], sectorsize))
+			memcpy(parity, pointers[rbio->scrubp], sectorsize);
 		else
 			/* Parity is right, needn't writeback */
 			bitmap_clear(rbio->dbitmap, sectornr, 1);
@@ -2559,10 +2565,12 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	}
 
 	kunmap_local(pointers[nr_data]);
-	__free_page(p_page);
-	if (q_page) {
+	__free_page(p_sector.page);
+	p_sector.page = NULL;
+	if (q_sector.page) {
 		kunmap_local(pointers[rbio->real_stripes - 1]);
-		__free_page(q_page);
+		__free_page(q_sector.page);
+		q_sector.page = NULL;
 	}
 
 writeback:
Signature · 1 line
2.35.1
author
#23 lore raw

PATCH v2 9/17 btrfs: make __raid_recover_endio_io() subpage compatibable

In reply to Qu Wenruo

This involves:

- Use sector_ptr interface to grab the pointers

- Add sector->pgoff to pointers[]

- Rebuild data using sectorsize instead of PAGE_SIZE

- Use memcpy() to replace copy_page()

---
 fs/btrfs/raid56.c | 52 ++++++++++++++++++++++++++---------------------
 1 file changed, 29 insertions(+), 23 deletions(-)
fs/btrfs/raid56.c +29 -23
index 6b2d75a09896..075b996040ba 100644
@@ -1929,14 +1929,18 @@ int raid56_parity_write(struct bio *bio, struct btrfs_io_context *bioc,
  */
 static void __raid_recover_end_io(struct btrfs_raid_bio *rbio)
 {
-	int pagenr, stripe;
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
+	int sectornr, stripe;
 	void **pointers;
 	void **unmap_array;
 	int faila = -1, failb = -1;
-	struct page *page;
 	blk_status_t err;
 	int i;
 
+	/*
+	 * This array stores the pointer for each sector, thus it has the extra
+	 * pgoff value added from each sector
+	 */
 	pointers = kcalloc(rbio->real_stripes, sizeof(void *), GFP_NOFS);
 	if (!pointers) {
 		err = BLK_STS_RESOURCE;
@@ -1965,43 +1969,44 @@ static void __raid_recover_end_io(struct btrfs_raid_bio *rbio)
 
 	index_rbio_pages(rbio);
 
-	for (pagenr = 0; pagenr < rbio->stripe_npages; pagenr++) {
+	for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
+		struct sector_ptr *sector;
+
 		/*
 		 * Now we just use bitmap to mark the horizontal stripes in
 		 * which we have data when doing parity scrub.
 		 */
 		if (rbio->operation == BTRFS_RBIO_PARITY_SCRUB &&
-		    !test_bit(pagenr, rbio->dbitmap))
+		    !test_bit(sectornr, rbio->dbitmap))
 			continue;
 
 		/*
-		 * Setup our array of pointers with pages from each stripe
+		 * Setup our array of pointers with sectors from each stripe
 		 *
 		 * NOTE: store a duplicate array of pointers to preserve the
 		 * pointer order
 		 */
 		for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
 			/*
-			 * if we're rebuilding a read, we have to use
+			 * If we're rebuilding a read, we have to use
 			 * pages from the bio list
 			 */
 			if ((rbio->operation == BTRFS_RBIO_READ_REBUILD ||
 			     rbio->operation == BTRFS_RBIO_REBUILD_MISSING) &&
 			    (stripe == faila || stripe == failb)) {
-				page = page_in_rbio(rbio, stripe, pagenr, 0);
+				sector = sector_in_rbio(rbio, stripe, sectornr, 0);
 			} else {
-				page = rbio_stripe_page(rbio, stripe, pagenr);
+				sector = rbio_stripe_sector(rbio, stripe, sectornr);
 			}
-			pointers[stripe] = kmap_local_page(page);
+			ASSERT(sector->page);
+			pointers[stripe] = kmap_local_page(sector->page) +
+					   sector->pgoff;
 			unmap_array[stripe] = pointers[stripe];
 		}
 
-		/* all raid6 handling here */
+		/* All raid6 handling here */
 		if (rbio->bioc->map_type & BTRFS_BLOCK_GROUP_RAID6) {
-			/*
-			 * single failure, rebuild from parity raid5
-			 * style
-			 */
+			/* Single failure, rebuild from parity raid5 style */
 			if (failb < 0) {
 				if (faila == rbio->nr_data) {
 					/*
@@ -2044,10 +2049,10 @@ static void __raid_recover_end_io(struct btrfs_raid_bio *rbio)
 
 			if (rbio->bioc->raid_map[failb] == RAID5_P_STRIPE) {
 				raid6_datap_recov(rbio->real_stripes,
-						  PAGE_SIZE, faila, pointers);
+						  sectorsize, faila, pointers);
 			} else {
 				raid6_2data_recov(rbio->real_stripes,
-						  PAGE_SIZE, faila, failb,
+						  sectorsize, faila, failb,
 						  pointers);
 			}
 		} else {
@@ -2057,7 +2062,8 @@ static void __raid_recover_end_io(struct btrfs_raid_bio *rbio)
 			BUG_ON(failb != -1);
 pstripe:
 			/* Copy parity block into failed block to start with */
-			copy_page(pointers[faila], pointers[rbio->nr_data]);
+			memcpy(pointers[faila], pointers[rbio->nr_data],
+			       sectorsize);
 
 			/* rearrange the pointer array */
 			p = pointers[faila];
@@ -2066,7 +2072,7 @@ static void __raid_recover_end_io(struct btrfs_raid_bio *rbio)
 			pointers[rbio->nr_data - 1] = p;
 
 			/* xor in the rest */
-			run_xor(pointers, rbio->nr_data - 1, PAGE_SIZE);
+			run_xor(pointers, rbio->nr_data - 1, sectorsize);
 		}
 		/* if we're doing this rebuild as part of an rmw, go through
 		 * and set all of our private rbio pages in the
@@ -2075,14 +2081,14 @@ static void __raid_recover_end_io(struct btrfs_raid_bio *rbio)
 		 * other endio functions will fiddle the uptodate bits
 		 */
 		if (rbio->operation == BTRFS_RBIO_WRITE) {
-			for (i = 0;  i < rbio->stripe_npages; i++) {
+			for (i = 0;  i < rbio->stripe_nsectors; i++) {
 				if (faila != -1) {
-					page = rbio_stripe_page(rbio, faila, i);
-					SetPageUptodate(page);
+					sector = rbio_stripe_sector(rbio, faila, i);
+					sector->uptodate = 1;
 				}
 				if (failb != -1) {
-					page = rbio_stripe_page(rbio, failb, i);
-					SetPageUptodate(page);
+					sector = rbio_stripe_sector(rbio, failb, i);
+					sector->uptodate = 1;
 				}
 			}
 		}
Signature · 1 line
2.35.1
author
#24 lore raw

PATCH v2 10/17 btrfs: make finish_rmw() subpage compatible

In reply to Qu Wenruo

With this function converted to subpage compatible sector interfaces, the following helper functions can be removed:

- rbio_stripe_page()
- rbio_pstripe_page()
- rbio_qstripe_page()
- page_in_rbio()

---
 fs/btrfs/raid56.c | 108 ++++++++++++++--------------------------------
 1 file changed, 32 insertions(+), 76 deletions(-)
fs/btrfs/raid56.c +32 -76
index 075b996040ba..928359840e8e 100644
@@ -681,39 +681,26 @@ static struct sector_ptr *rbio_stripe_sector(const struct btrfs_raid_bio *rbio,
 							      sector_nr)];
 }
 
-static int rbio_stripe_page_index(struct btrfs_raid_bio *rbio, int stripe,
-				  int index)
+/* Helper to grab a sector inside P stripe */
+static struct sector_ptr *rbio_pstripe_sector(const struct btrfs_raid_bio *rbio,
+					      unsigned int sector_nr)
 {
-	return stripe * rbio->stripe_npages + index;
+	return rbio_stripe_sector(rbio, rbio->nr_data, sector_nr);
 }
 
-/*
- * these are just the pages from the rbio array, not from anything
- * the FS sent down to us
- */
-static struct page *rbio_stripe_page(struct btrfs_raid_bio *rbio, int stripe,
-				     int index)
+/* Helper to grab a sector inside Q stripe, return NULL if not RAID6 */
+static struct sector_ptr *rbio_qstripe_sector(const struct btrfs_raid_bio *rbio,
+					      unsigned int sector_nr)
 {
-	return rbio->stripe_pages[rbio_stripe_page_index(rbio, stripe, index)];
-}
-
-/*
- * helper to index into the pstripe
- */
-static struct page *rbio_pstripe_page(struct btrfs_raid_bio *rbio, int index)
-{
-	return rbio_stripe_page(rbio, rbio->nr_data, index);
+	if (rbio->nr_data + 1 == rbio->real_stripes)
+		return NULL;
+	return rbio_stripe_sector(rbio, rbio->nr_data + 1, sector_nr);
 }
 
-/*
- * helper to index into the qstripe, returns null
- * if there is no qstripe
- */
-static struct page *rbio_qstripe_page(struct btrfs_raid_bio *rbio, int index)
+static int rbio_stripe_page_index(struct btrfs_raid_bio *rbio, int stripe,
+				  int index)
 {
-	if (rbio->nr_data + 1 == rbio->real_stripes)
-		return NULL;
-	return rbio_stripe_page(rbio, rbio->nr_data + 1, index);
+	return stripe * rbio->stripe_npages + index;
 }
 
 /*
@@ -1030,40 +1017,6 @@ static struct sector_ptr *sector_in_rbio(struct btrfs_raid_bio *rbio,
 	return &rbio->stripe_sectors[index];
 }
 
-/*
- * the read/modify/write code wants to use the original bio for
- * any pages it included, and then use the rbio for everything
- * else.  This function decides if a given index (stripe number)
- * and page number in that stripe fall inside the original bio
- * or the rbio.
- *
- * if you set bio_list_only, you'll get a NULL back for any ranges
- * that are outside the bio_list
- *
- * This doesn't take any refs on anything, you get a bare page pointer
- * and the caller must bump refs as required.
- *
- * You must call index_rbio_pages once before you can trust
- * the answers from this function.
- */
-static struct page *page_in_rbio(struct btrfs_raid_bio *rbio,
-				 int index, int pagenr, int bio_list_only)
-{
-	int chunk_page;
-	struct page *p = NULL;
-
-	chunk_page = index * (rbio->stripe_len >> PAGE_SHIFT) + pagenr;
-
-	spin_lock_irq(&rbio->bio_list_lock);
-	p = rbio->bio_pages[chunk_page];
-	spin_unlock_irq(&rbio->bio_list_lock);
-
-	if (p || bio_list_only)
-		return p;
-
-	return rbio->stripe_pages[chunk_page];
-}
-
 /*
  * allocation and initial setup for the btrfs_raid_bio.  Not
  * this does not allocate any pages for rbio->pages.
@@ -1333,6 +1286,7 @@ static void index_rbio_pages(struct btrfs_raid_bio *rbio)
 static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 {
 	struct btrfs_io_context *bioc = rbio->bioc;
+	const u32 sectorsize = bioc->fs_info->sectorsize;
 	void **pointers = rbio->finish_pointers;
 	int nr_data = rbio->nr_data;
 	int stripe;
@@ -1381,34 +1335,36 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)
 		clear_bit(RBIO_CACHE_READY_BIT, &rbio->flags);
 
 	for (sectornr = 0; sectornr < rbio->stripe_nsectors; sectornr++) {
-		struct page *p;
-		/* first collect one page from each data stripe */
+		struct sector_ptr *sector;
+
+		/* First collect one sector from each data stripe */
 		for (stripe = 0; stripe < nr_data; stripe++) {
-			p = page_in_rbio(rbio, stripe, sectornr, 0);
-			pointers[stripe] = kmap_local_page(p);
+			sector = sector_in_rbio(rbio, stripe, sectornr, 0);
+			pointers[stripe] = kmap_local_page(sector->page) +
+					   sector->pgoff;
 		}
 
-		/* then add the parity stripe */
-		p = rbio_pstripe_page(rbio, sectornr);
-		SetPageUptodate(p);
-		pointers[stripe++] = kmap_local_page(p);
+		/* Then add the parity stripe */
+		sector = rbio_pstripe_sector(rbio, sectornr);
+		sector->uptodate = 1;
+		pointers[stripe++] = kmap_local_page(sector->page) + sector->pgoff;
 
 		if (has_qstripe) {
-
 			/*
-			 * raid6, add the qstripe and call the
+			 * RAID6, add the qstripe and call the
 			 * library function to fill in our p/q
 			 */
-			p = rbio_qstripe_page(rbio, sectornr);
-			SetPageUptodate(p);
-			pointers[stripe++] = kmap_local_page(p);
+			sector = rbio_qstripe_sector(rbio, sectornr);
+			sector->uptodate = 1;
+			pointers[stripe++] = kmap_local_page(sector->page) +
+					     sector->pgoff;
 
-			raid6_call.gen_syndrome(rbio->real_stripes, PAGE_SIZE,
+			raid6_call.gen_syndrome(rbio->real_stripes, sectorsize,
 						pointers);
 		} else {
 			/* raid5 */
-			copy_page(pointers[nr_data], pointers[0]);
-			run_xor(pointers + 1, nr_data - 1, PAGE_SIZE);
+			memcpy(pointers[nr_data], pointers[0], sectorsize);
+			run_xor(pointers + 1, nr_data - 1, sectorsize);
 		}
 		for (stripe = stripe - 1; stripe >= 0; stripe--)
 			kunmap_local(pointers[stripe]);
Signature · 1 line
2.35.1
author
#25 lore raw

PATCH v2 11/17 btrfs: open-code rbio_stripe_page_index()

In reply to Qu Wenruo

There is only one caller for that helper now, and we're definitely fine to open-code it.

---
 fs/btrfs/raid56.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
fs/btrfs/raid56.c +1 -7
index 928359840e8e..5d383f894eb9 100644
@@ -697,12 +697,6 @@ static struct sector_ptr *rbio_qstripe_sector(const struct btrfs_raid_bio *rbio,
 	return rbio_stripe_sector(rbio, rbio->nr_data + 1, sector_nr);
 }
 
-static int rbio_stripe_page_index(struct btrfs_raid_bio *rbio, int stripe,
-				  int index)
-{
-	return stripe * rbio->stripe_npages + index;
-}
-
 /*
  * The first stripe in the table for a logical address
  * has the lock.  rbios are added in one of three ways:
@@ -1117,7 +1111,7 @@ static int alloc_rbio_pages(struct btrfs_raid_bio *rbio)
 /* only allocate pages for p/q stripes */
 static int alloc_rbio_parity_pages(struct btrfs_raid_bio *rbio)
 {
-	int data_pages = rbio_stripe_page_index(rbio, rbio->nr_data, 0);
+	int data_pages = rbio->nr_data * rbio->stripe_npages;
 	int ret;
 
 	ret = btrfs_alloc_page_array(rbio->nr_pages - data_pages,
Signature · 1 line
2.35.1
author
#26 lore raw

PATCH v2 12/17 btrfs: make raid56_add_scrub_pages() subpage compatible

In reply to Qu Wenruo

This requires one extra parameter @pgoff for the function.

In the current code base, scrub is still one page per sector, thus the new parameter will always be 0.

It needs the extra subpage scrub optimization code to fully take advantage.

---
 fs/btrfs/raid56.c | 10 ++++++----
 fs/btrfs/raid56.h |  2 +-
 fs/btrfs/scrub.c  |  6 +++++-
 3 files changed, 12 insertions(+), 6 deletions(-)
fs/btrfs/raid56.c +6 -4
index 5d383f894eb9..5f89ff3963c2 100644
@@ -2379,17 +2379,19 @@ struct btrfs_raid_bio *raid56_parity_alloc_scrub_rbio(struct bio *bio,
 
 /* Used for both parity scrub and missing. */
 void raid56_add_scrub_pages(struct btrfs_raid_bio *rbio, struct page *page,
-			    u64 logical)
+			    unsigned int pgoff, u64 logical)
 {
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
 	int stripe_offset;
 	int index;
 
 	ASSERT(logical >= rbio->bioc->raid_map[0]);
-	ASSERT(logical + PAGE_SIZE <= rbio->bioc->raid_map[0] +
+	ASSERT(logical + sectorsize <= rbio->bioc->raid_map[0] +
 				rbio->stripe_len * rbio->nr_data);
 	stripe_offset = (int)(logical - rbio->bioc->raid_map[0]);
-	index = stripe_offset >> PAGE_SHIFT;
-	rbio->bio_pages[index] = page;
+	index = stripe_offset / sectorsize;
+	rbio->bio_sectors[index].page = page;
+	rbio->bio_sectors[index].pgoff = pgoff;
 }
 
 /*
fs/btrfs/raid56.h +1 -1
index fb35ae157b02..b5bc0feb3401 100644
@@ -36,7 +36,7 @@ int raid56_parity_write(struct bio *bio, struct btrfs_io_context *bioc,
 			u32 stripe_len);
 
 void raid56_add_scrub_pages(struct btrfs_raid_bio *rbio, struct page *page,
-			    u64 logical);
+			    unsigned int pgoff, u64 logical);
 
 struct btrfs_raid_bio *raid56_parity_alloc_scrub_rbio(struct bio *bio,
 				struct btrfs_io_context *bioc, u32 stripe_len,
fs/btrfs/scrub.c +5 -1
index 13ba458c080c..ccc2cb869cca 100644
@@ -2205,7 +2205,11 @@ static void scrub_missing_raid56_pages(struct scrub_block *sblock)
 	for (i = 0; i < sblock->sector_count; i++) {
 		struct scrub_sector *sector = sblock->sectors[i];
 
-		raid56_add_scrub_pages(rbio, sector->page, sector->logical);
+		/*
+		 * For now, our scrub is still one page per-sector, so pgoff
+		 * is always 0.
+		 */
+		raid56_add_scrub_pages(rbio, sector->page, 0, sector->logical);
 	}
 
 	btrfs_init_work(&sblock->work, scrub_missing_raid56_worker, NULL, NULL);
Signature · 1 line
2.35.1
author
#27 lore raw

PATCH v2 13/17 btrfs: remove btrfs_raid_bio::bio_pages array

In reply to Qu Wenruo

The functionality is completely replaced by the new bio_sectors member, now it's time to remove the old member.

---
 fs/btrfs/raid56.c | 38 +++-----------------------------------
 1 file changed, 3 insertions(+), 35 deletions(-)
fs/btrfs/raid56.c +3 -35
index 5f89ff3963c2..af8ba1aff682 100644
@@ -178,12 +178,6 @@ struct btrfs_raid_bio {
 	/* Pointers to the sectors in the bio_list, for faster lookup */
 	struct sector_ptr *bio_sectors;
 
-	/*
-	 * pointers to the pages in the bio_list.  Stored
-	 * here for faster lookup
-	 */
-	struct page **bio_pages;
-
 	/*
 	 * For subpage support, we need to map each sector to above
 	 * stripe_pages.
@@ -265,7 +259,7 @@ int btrfs_alloc_stripe_hash_table(struct btrfs_fs_info *info)
 
 /*
  * caching an rbio means to copy anything from the
- * bio_pages array into the stripe_pages array.  We
+ * bio_sectors array into the stripe_pages array.  We
  * use the page uptodate bit in the stripe cache array
  * to indicate if it has valid data
  *
@@ -281,14 +275,6 @@ static void cache_rbio_pages(struct btrfs_raid_bio *rbio)
 	if (ret)
 		return;
 
-	for (i = 0; i < rbio->nr_pages; i++) {
-		if (!rbio->bio_pages[i])
-			continue;
-
-		copy_highpage(rbio->stripe_pages[i], rbio->bio_pages[i]);
-		SetPageUptodate(rbio->stripe_pages[i]);
-	}
-
 	for (i = 0; i < rbio->nr_sectors; i++) {
 		/* Some range not covered by bio (partial write), skip it */
 		if (!rbio->bio_sectors[i].page)
@@ -1064,7 +1050,7 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 	atomic_set(&rbio->stripes_pending, 0);
 
 	/*
-	 * the stripe_pages, bio_pages, etc arrays point to the extra
+	 * The stripe_pages, bio_sectors, etc arrays point to the extra
 	 * memory we allocated past the end of the rbio
 	 */
 	p = rbio + 1;
@@ -1073,7 +1059,6 @@ static struct btrfs_raid_bio *alloc_rbio(struct btrfs_fs_info *fs_info,
 		p = (unsigned char *)p + sizeof(*(ptr)) * (count);	\
 	} while (0)
 	CONSUME_ALLOC(rbio->stripe_pages, num_pages);
-	CONSUME_ALLOC(rbio->bio_pages, num_pages);
 	CONSUME_ALLOC(rbio->bio_sectors, num_sectors);
 	CONSUME_ALLOC(rbio->stripe_sectors, num_sectors);
 	CONSUME_ALLOC(rbio->finish_pointers, real_stripes);
@@ -1234,7 +1219,7 @@ static void index_one_bio(struct btrfs_raid_bio *rbio, struct bio *bio)
 }
 
 /*
- * helper function to walk our bio list and populate the bio_pages array with
+ * Helper function to walk our bio list and populate the bio_sectors array with
  * the result.  This seems expensive, but it is faster than constantly
  * searching through the bio list as we setup the IO in finish_rmw or stripe
  * reconstruction.
@@ -1244,25 +1229,8 @@ static void index_one_bio(struct btrfs_raid_bio *rbio, struct bio *bio)
 static void index_rbio_pages(struct btrfs_raid_bio *rbio)
 {
 	struct bio *bio;
-	u64 start;
-	unsigned long stripe_offset;
-	unsigned long page_index;
 
 	spin_lock_irq(&rbio->bio_list_lock);
-	bio_list_for_each(bio, &rbio->bio_list) {
-		struct bio_vec bvec;
-		struct bvec_iter iter;
-		int i = 0;
-
-		start = bio->bi_iter.bi_sector << 9;
-		stripe_offset = start - rbio->bioc->raid_map[0];
-		page_index = stripe_offset >> PAGE_SHIFT;
-
-		bio_for_each_segment(bvec, bio, iter) {
-			rbio->bio_pages[page_index + i] = bvec.bv_page;
-			i++;
-		}
-	}
 	bio_list_for_each(bio, &rbio->bio_list)
 		index_one_bio(rbio, bio);
 
Signature · 1 line
2.35.1
author
#28 lore raw

PATCH v2 14/17 btrfs: make set_bio_pages_uptodate() subpage compatible

In reply to Qu Wenruo

Unlike previous code, we can not directly set PageUptodate for stripe pages now.

Instead we have to iterate through all the sectors and set SECTOR_UPTODATE flag there.

Thus this patch introduced a new helper, find_stripe_sector(), to do the work.

---
 fs/btrfs/raid56.c | 45 +++++++++++++++++++++++++++++++++++++++------
 1 file changed, 39 insertions(+), 6 deletions(-)
fs/btrfs/raid56.c +39 -6
index af8ba1aff682..8de5294de7a5 100644
@@ -1491,19 +1491,52 @@ static int fail_bio_stripe(struct btrfs_raid_bio *rbio,
 	return fail_rbio_index(rbio, failed);
 }
 
+/*
+ * For subpage case, we can no longer set page Uptodate directly for
+ * stripe_pages[], thus we need to locate the sector.
+ */
+static struct sector_ptr *find_stripe_sector(struct btrfs_raid_bio *rbio,
+					     struct page *page,
+					     unsigned int pgoff)
+{
+	int i;
+
+	for (i = 0; i < rbio->nr_sectors; i++) {
+		struct sector_ptr *sector = &rbio->stripe_sectors[i];
+
+		if (sector->page == page && sector->pgoff == pgoff)
+			return sector;
+	}
+	return NULL;
+}
+
 /*
  * this sets each page in the bio uptodate.  It should only be used on private
  * rbio pages, nothing that comes in from the higher layers
  */
-static void set_bio_pages_uptodate(struct bio *bio)
+static void set_bio_pages_uptodate(struct btrfs_raid_bio *rbio,
+				   struct bio *bio)
 {
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
 	struct bio_vec *bvec;
 	struct bvec_iter_all iter_all;
 
 	ASSERT(!bio_flagged(bio, BIO_CLONED));
 
-	bio_for_each_segment_all(bvec, bio, iter_all)
-		SetPageUptodate(bvec->bv_page);
+	bio_for_each_segment_all(bvec, bio, iter_all) {
+		struct sector_ptr *sector;
+		int pgoff;
+
+		for (pgoff = bvec->bv_offset;
+		     pgoff - bvec->bv_offset < bvec->bv_len;
+		     pgoff += sectorsize) {
+			sector = find_stripe_sector(rbio, bvec->bv_page, pgoff);
+			ASSERT(sector);
+			if (sector)
+				sector->uptodate = 1;
+		}
+
+	}
 }
 
 /*
@@ -1521,7 +1554,7 @@ static void raid_rmw_end_io(struct bio *bio)
 	if (bio->bi_status)
 		fail_bio_stripe(rbio, bio);
 	else
-		set_bio_pages_uptodate(bio);
+		set_bio_pages_uptodate(rbio, bio);
 
 	bio_put(bio);
 
@@ -2079,7 +2112,7 @@ static void raid_recover_end_io(struct bio *bio)
 	if (bio->bi_status)
 		fail_bio_stripe(rbio, bio);
 	else
-		set_bio_pages_uptodate(bio);
+		set_bio_pages_uptodate(rbio, bio);
 	bio_put(bio);
 
 	if (!atomic_dec_and_test(&rbio->stripes_pending))
@@ -2637,7 +2670,7 @@ static void raid56_parity_scrub_end_io(struct bio *bio)
 	if (bio->bi_status)
 		fail_bio_stripe(rbio, bio);
 	else
-		set_bio_pages_uptodate(bio);
+		set_bio_pages_uptodate(rbio, bio);
 
 	bio_put(bio);
 
Signature · 1 line
2.35.1
author
#29 lore raw

PATCH v2 15/17 btrfs: make steal_rbio() subpage compatible

In reply to Qu Wenruo

Function steal_rbio() will taking all the uptodate pages from the source rbio to destination rbio.

With the new stripe_sectors[] array, we also need to do the extra check:

- Check sector::flags to make sure the full page is uptodate Now we don't use PageUptodate flag for subpage cases to indicate if the page is uptodate.

Instead we need to check all the sectors belong to the page to be sure about whether it's full page uptodate.

So here we introduce a new helper, full_page_sectors_uptodate() to do the check.

- Update rbio::stripe_sectors[] to use the new page pointer We only need to change the page pointer, no need to change anything else.

---
 fs/btrfs/raid56.c | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)
fs/btrfs/raid56.c +24 -3
index 8de5294de7a5..d0f9bee0c10d 100644
@@ -309,6 +309,25 @@ static int rbio_bucket(struct btrfs_raid_bio *rbio)
 	return hash_64(num >> 16, BTRFS_STRIPE_HASH_TABLE_BITS);
 }
 
+static bool full_page_sectors_uptodate(struct btrfs_raid_bio *rbio,
+				       unsigned int page_nr)
+{
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
+	const u32 sectors_per_page = PAGE_SIZE / sectorsize;
+	bool uptodate = true;
+	int i;
+
+	ASSERT(page_nr < rbio->nr_pages);
+
+	for (i = sectors_per_page * page_nr;
+	     i < sectors_per_page * page_nr + sectors_per_page; i++) {
+		if (!rbio->stripe_sectors[i].uptodate) {
+			uptodate = false;
+			break;
+		}
+	}
+	return uptodate;
+}
 /*
  * Update the stripe_sectors[] array to use correct page and pgoff
  *
@@ -330,8 +349,11 @@ static void index_stripe_sectors(struct btrfs_raid_bio *rbio)
 }
 
 /*
- * stealing an rbio means taking all the uptodate pages from the stripe
+ * Stealing an rbio means taking all the uptodate pages from the stripe
  * array in the source rbio and putting them into the destination rbio
+ *
+ * This will also update the involved stripe_sectors[] which are referring to
+ * the old pages.
  */
 static void steal_rbio(struct btrfs_raid_bio *src, struct btrfs_raid_bio *dest)
 {
@@ -344,9 +366,8 @@ static void steal_rbio(struct btrfs_raid_bio *src, struct btrfs_raid_bio *dest)
 
 	for (i = 0; i < dest->nr_pages; i++) {
 		s = src->stripe_pages[i];
-		if (!s || !PageUptodate(s)) {
+		if (!s || !full_page_sectors_uptodate(src, i))
 			continue;
-		}
 
 		d = dest->stripe_pages[i];
 		if (d)
Signature · 1 line
2.35.1
author
#30 lore raw

PATCH v2 16/17 btrfs: make alloc_rbio_essential_pages() subpage compatible

In reply to Qu Wenruo

The non-compatible part is only the bitmap iteration part, now the bitmap size is extended to rbio::stripe_nsectors, not the old rbio::stripe_npages.

Since we're here, also slightly improve the function by:

- Rename @i to @stripe
- Rename @bit to @sectornr
- Move @page and @index into the inner loop

---
 fs/btrfs/raid56.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
fs/btrfs/raid56.c +9 -7
index d0f9bee0c10d..ef3d3b67098b 100644
@@ -2422,14 +2422,16 @@ void raid56_add_scrub_pages(struct btrfs_raid_bio *rbio, struct page *page,
  */
 static int alloc_rbio_essential_pages(struct btrfs_raid_bio *rbio)
 {
-	int i;
-	int bit;
-	int index;
-	struct page *page;
+	const u32 sectorsize = rbio->bioc->fs_info->sectorsize;
+	int stripe;
+	int sectornr;
+
+	for_each_set_bit(sectornr, rbio->dbitmap, rbio->stripe_nsectors) {
+		for (stripe = 0; stripe < rbio->real_stripes; stripe++) {
+			struct page *page;
+			int index = (stripe * rbio->stripe_nsectors + sectornr) *
+				    sectorsize >> PAGE_SHIFT;
 
-	for_each_set_bit(bit, rbio->dbitmap, rbio->stripe_npages) {
-		for (i = 0; i < rbio->real_stripes; i++) {
-			index = i * rbio->stripe_npages + bit;
 			if (rbio->stripe_pages[index])
 				continue;
 
Signature · 1 line
2.35.1
author
#31 lore raw

PATCH v2 17/17 btrfs: enable subpage support for RAID56

In reply to Qu Wenruo

Now the btrfs RAID56 infrastructure has migrated to use sector_ptr interface, it should be safe to enable subpage support for btrfs RAID56.

---
 fs/btrfs/disk-io.c | 8 --------
 fs/btrfs/raid56.c  | 6 ------
 fs/btrfs/volumes.c | 7 -------
 3 files changed, 21 deletions(-)
fs/btrfs/disk-io.c +0 -8
index 2bc867d35308..e816943ddb7a 100644
@@ -3682,14 +3682,6 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device
 		btrfs_warn(fs_info,
 		"read-write for sector size %u with page size %lu is experimental",
 			   sectorsize, PAGE_SIZE);
-		if (btrfs_super_incompat_flags(fs_info->super_copy) &
-			BTRFS_FEATURE_INCOMPAT_RAID56) {
-			btrfs_err(fs_info,
-		"RAID56 is not yet supported for sector size %u with page size %lu",
-				sectorsize, PAGE_SIZE);
-			err = -EINVAL;
-			goto fail_alloc;
-		}
 		subpage_info = kzalloc(sizeof(*subpage_info), GFP_KERNEL);
 		if (!subpage_info)
 			goto fail_alloc;
fs/btrfs/raid56.c +0 -6
index ef3d3b67098b..97a92576a163 100644
@@ -1158,9 +1158,6 @@ static int rbio_add_io_sector(struct btrfs_raid_bio *rbio,
 	ASSERT(sector_nr >= 0 && sector_nr < rbio->stripe_nsectors);
 	ASSERT(sector->page);
 
-	/* We don't yet support subpage, thus pgoff should always be 0 */
-	ASSERT(sector->pgoff == 0);
-
 	stripe = &rbio->bioc->stripes[stripe_nr];
 	disk_start = stripe->physical + sector_nr * sectorsize;
 
@@ -2385,9 +2382,6 @@ struct btrfs_raid_bio *raid56_parity_alloc_scrub_rbio(struct bio *bio,
 	}
 	ASSERT(i < rbio->real_stripes);
 
-	/* Now we just support the sectorsize equals to page size */
-	ASSERT(fs_info->sectorsize == PAGE_SIZE);
-	ASSERT(rbio->stripe_npages == stripe_nsectors);
 	bitmap_copy(rbio->dbitmap, dbitmap, stripe_nsectors);
 
 	/*
fs/btrfs/volumes.c +0 -7
index 93072a090fdd..1b75cde5a267 100644
@@ -4063,13 +4063,6 @@ static inline int validate_convert_profile(struct btrfs_fs_info *fs_info,
 	if (!(bargs->flags & BTRFS_BALANCE_ARGS_CONVERT))
 		return true;
 
-	if (fs_info->sectorsize < PAGE_SIZE &&
-		bargs->target & BTRFS_BLOCK_GROUP_RAID56_MASK) {
-		btrfs_err(fs_info,
-		"RAID56 is not yet supported for sectorsize %u with page size %lu",
-			  fs_info->sectorsize, PAGE_SIZE);
-		return false;
-	}
 	/* Profile is valid and does not have bits outside of the allowed set */
 	if (alloc_profile_is_valid(bargs->target, 1) &&
 	    (bargs->target & ~allowed) == 0)
Signature · 1 line
2.35.1
#32 lore raw
In reply to Qu Wenruo
Quoted 36 lines · Qu Wenruo

The branch can be fetched from github, based on latest misc-next branch (with bio and memory allocation refactors):
https://github.com/adam900710/linux/tree/subpage_raid56

[CHANGELOG]
v2:
- Rebased to latest misc-next
There are several conflicts caused by bio interface change and page allocation update.

- A new patch to reduce the width of @stripe_len to u32 Currently @stripe_len is fixed to 64K, and even in the future we choose to enlarge the value, I see no reason to go beyond 4G for stripe length.

Thus change it u32 to avoid some u64-divided-by-u32 situations.

This will reduce memory usage for map_lookup (which has a lifespan as long as the mounted fs) and btrfs_io_geometry (which only has a very short lifespan, mostly bounded to bio).

Furthermore, add some extra alignment check and use right bit shift to replace involved division to avoid possible problems on 32bit systems.

- Pack sector_ptr::pgoff and sector_ptr::uptodate into one u32 This will reduce memory usage and reduce unaligned memory access

Please note that, even with it packed, we still have a 4 bytes padding (it's u64 + u32, thus not perfectly aligned).
Without packed attribute, it will cost more memory usage anyway.

- Call kunmap_local() using address with pgoff
As it can handle it without problem, no need to bother extra search just for pgoff.

Quoted 5 lines

- Use "= { 0 }" for structure initialization

- Reduce comment updates to minimal
If one comment line is not really touched, then don't touch it just to fix some bad styles.

v2 updated in for-next, I had a local branch with more changes so I transferred the changes manually.

#33 lore raw
Quoted 37 lines · David Sterba
Quoted 36 lines · Qu Wenruo

The branch can be fetched from github, based on latest misc-next branch (with bio and memory allocation refactors):
https://github.com/adam900710/linux/tree/subpage_raid56

[CHANGELOG]
v2:
- Rebased to latest misc-next
There are several conflicts caused by bio interface change and page allocation update.

- A new patch to reduce the width of @stripe_len to u32 Currently @stripe_len is fixed to 64K, and even in the future we choose to enlarge the value, I see no reason to go beyond 4G for stripe length.

Thus change it u32 to avoid some u64-divided-by-u32 situations.

This will reduce memory usage for map_lookup (which has a lifespan as long as the mounted fs) and btrfs_io_geometry (which only has a very short lifespan, mostly bounded to bio).

Furthermore, add some extra alignment check and use right bit shift to replace involved division to avoid possible problems on 32bit systems.

- Pack sector_ptr::pgoff and sector_ptr::uptodate into one u32 This will reduce memory usage and reduce unaligned memory access

Please note that, even with it packed, we still have a 4 bytes padding (it's u64 + u32, thus not perfectly aligned).
Without packed attribute, it will cost more memory usage anyway.

- Call kunmap_local() using address with pgoff
As it can handle it without problem, no need to bother extra search just for pgoff.

Quoted 8 lines
Quoted 5 lines

- Use "= { 0 }" for structure initialization

- Reduce comment updates to minimal
If one comment line is not really touched, then don't touch it just to fix some bad styles.

v2 updated in for-next, I had a local branch with more changes so I transferred the changes manually.

Now moved to misc-next as it's an isolated functionality and should not affect anything else. Thanks.