Skip to contentSkip to conversation

Did immutable bvecs accidentally break stable page writes?

DISCUSSION Mel Gorman 3 participants 4 messages Sep 19, 2013 – Sep 22, 2013

lore
Share & tools
lore

Mel Gorman

author
#0 lore raw

Commit ffecfd1a (block: optionally snapshot page contents to provide stable pages during write) uses bounce buffers for stable page writes in jbd and ext3. Simplistically, __blk_queue_bounce takes a force parameter that is used when pages must be snapshot.

Commit 6bc454d1 (bounce: Refactor __blk_queue_bounce to not use bi_io_vec) refactored __blk_queue_bounce and now the start of the function looks like this

static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
mempool_t *pool, int force)

{
	struct bio *bio;
	int rw = bio_data_dir(*bio_orig);
	struct bio_vec *to, *from;
	unsigned i;

	bio_for_each_segment(from, *bio_orig, i)
		if (page_to_pfn(from->bv_page) > queue_bounce_pfn(q))
			goto bounce;

	return;
bounce:
	bio = bio_clone_bioset(*bio_orig, GFP_NOIO, fs_bio_set);

	bio_for_each_segment_all(to, bio, i) {
		struct page *page = to->bv_page;

		if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force)
			continue;

Note that the first bio_for_each_segment is completely ignoring the force parameter and hence snapshotting. This is particularly problematic for ext3 which forces the use of MS_SNAP_STABLE.

I have not actually reproduced any problem, this is just code inspection but it looks like commit 6bc454d1 broke ext3. Kent, why was the force paramter ignored in that commit?

Signature · 2 lines
Mel Gorman
SUSE Labs
author
#1 lore raw
Quoted 31 lines · Mel Gorman

Commit ffecfd1a (block: optionally snapshot page contents to provide stable pages during write) uses bounce buffers for stable page writes in jbd and ext3. Simplistically, __blk_queue_bounce takes a force parameter that is used when pages must be snapshot.

Commit 6bc454d1 (bounce: Refactor __blk_queue_bounce to not use bi_io_vec) refactored __blk_queue_bounce and now the start of the function looks like this

static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
mempool_t *pool, int force)

{
	struct bio *bio;
	int rw = bio_data_dir(*bio_orig);
	struct bio_vec *to, *from;
	unsigned i;

	bio_for_each_segment(from, *bio_orig, i)
		if (page_to_pfn(from->bv_page) > queue_bounce_pfn(q))
			goto bounce;

	return;
bounce:
	bio = bio_clone_bioset(*bio_orig, GFP_NOIO, fs_bio_set);

	bio_for_each_segment_all(to, bio, i) {
		struct page *page = to->bv_page;

		if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force)
			continue;
Quoted 4 lines

Note that the first bio_for_each_segment is completely ignoring the force parameter and hence snapshotting. This is particularly problematic for ext3 which forces the use of MS_SNAP_STABLE.

Which of course is no longer a problem for ext3 after commit 71368511 ("mm: make snapshotting pages for stable writes a per-bio operation). The folly of looking at a commit in isolation! I'm still curious why the force parameter is ignored when stable writes are required though.

Signature · 2 lines
Mel Gorman
SUSE Labs
#2 lore raw
Quoted 37 lines · Mel Gorman
Quoted 31 lines · Mel Gorman

Commit ffecfd1a (block: optionally snapshot page contents to provide stable pages during write) uses bounce buffers for stable page writes in jbd and ext3. Simplistically, __blk_queue_bounce takes a force parameter that is used when pages must be snapshot.

Commit 6bc454d1 (bounce: Refactor __blk_queue_bounce to not use bi_io_vec) refactored __blk_queue_bounce and now the start of the function looks like this

static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
mempool_t *pool, int force)

{
	struct bio *bio;
	int rw = bio_data_dir(*bio_orig);
	struct bio_vec *to, *from;
	unsigned i;

	bio_for_each_segment(from, *bio_orig, i)
		if (page_to_pfn(from->bv_page) > queue_bounce_pfn(q))
			goto bounce;

	return;
bounce:
	bio = bio_clone_bioset(*bio_orig, GFP_NOIO, fs_bio_set);

	bio_for_each_segment_all(to, bio, i) {
		struct page *page = to->bv_page;

		if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force)
			continue;
Quoted 4 lines

Note that the first bio_for_each_segment is completely ignoring the force parameter and hence snapshotting. This is particularly problematic for ext3 which forces the use of MS_SNAP_STABLE.

Quoted 4 lines

Which of course is no longer a problem for ext3 after commit 71368511 ("mm: make snapshotting pages for stable writes a per-bio operation). The folly of looking at a commit in isolation! I'm still curious why the force parameter is ignored when stable writes are required though.

I'm confused by this (second) statement. Mel, I think you were arguing that this is broken because BIO_SNAP_STABLE sets force=1, yet __blk_queue_bounce() ignores "force", which leads to the pages not being bounced.

So it /is/ a likely regression, and testing confirms that DIF+ext3 is broken. I believe that there should be a "if(force) goto bounce;" just after the variable declarations. I'll go write a patch and check though.

--D

Quoted 4 lines
Signature · 3 lines
-- 
Mel Gorman
SUSE Labs
#3 lore raw
In reply to Mel Gorman
Quoted 37 lines · Mel Gorman
Quoted 31 lines · Mel Gorman

Commit ffecfd1a (block: optionally snapshot page contents to provide stable pages during write) uses bounce buffers for stable page writes in jbd and ext3. Simplistically, __blk_queue_bounce takes a force parameter that is used when pages must be snapshot.

Commit 6bc454d1 (bounce: Refactor __blk_queue_bounce to not use bi_io_vec) refactored __blk_queue_bounce and now the start of the function looks like this

static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
mempool_t *pool, int force)

{
	struct bio *bio;
	int rw = bio_data_dir(*bio_orig);
	struct bio_vec *to, *from;
	unsigned i;

	bio_for_each_segment(from, *bio_orig, i)
		if (page_to_pfn(from->bv_page) > queue_bounce_pfn(q))
			goto bounce;

	return;
bounce:
	bio = bio_clone_bioset(*bio_orig, GFP_NOIO, fs_bio_set);

	bio_for_each_segment_all(to, bio, i) {
		struct page *page = to->bv_page;

		if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force)
			continue;
Quoted 4 lines

Note that the first bio_for_each_segment is completely ignoring the force parameter and hence snapshotting. This is particularly problematic for ext3 which forces the use of MS_SNAP_STABLE.

Quoted 4 lines

Which of course is no longer a problem for ext3 after commit 71368511 ("mm: make snapshotting pages for stable writes a per-bio operation). The folly of looking at a commit in isolation! I'm still curious why the force parameter is ignored when stable writes are required though.

This really looks like a bug to me. Since stable pages are currently
required only for drives supporting DIF/DIX noone has likely noticed. But still we should fix that.

Honza

PS: As an unrelated note first 8 digits of the commit SHA are not unique for that commit. Time to switch to longer prefixes? :)

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR