Skip to contentSkip to conversation

bcachefs: fix iov_iter count underflow on sub-block dio read

PATCH Superseded Brian Foster 3 participants 6 messages Feb 5, 2024 – Feb 15, 2024

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bcachefs: fix iov_iter count underflow on sub-block dio read

bch2_direct_IO_read() checks the request offset and size for sector alignment and then falls through to a couple calculations to shrink the size of the request based on the inode size. The problem is that these checks round up to the fs block size, which runs the risk of underflowing iter->count if the block size happens to be large enough. This is triggered by fstest generic/361 with a 4k block size, which subsequently leads to a crash.

After some discussion, the original purpose of the shorten logic in this path is not totally clear. It appears to be intended as an optimization of limited value, so simplify things and avoid the underflow problem by removing it.

---

Note that I left the ret variable name alone because it seemed to bother me less after we realized it is actually the dio return value.

Brian

 fs/bcachefs/fs-io-direct.c | 6 ------
 1 file changed, 6 deletions(-)
fs/bcachefs/fs-io-direct.c +0 -6
index e3b219e19e10..53f6e8a939d5 100644
@@ -72,7 +72,6 @@ static int bch2_direct_IO_read(struct kiocb *req, struct iov_iter *iter)
 	struct bio *bio;
 	loff_t offset = req->ki_pos;
 	bool sync = is_sync_kiocb(req);
-	size_t shorten;
 	ssize_t ret;
 
 	bch2_inode_opts_get(&opts, c, &inode->ei_inode);
@@ -87,9 +86,6 @@ static int bch2_direct_IO_read(struct kiocb *req, struct iov_iter *iter)
 	if (!ret)
 		return ret;
 
-	shorten = iov_iter_count(iter) - round_up(ret, block_bytes(c));
-	iter->count -= shorten;
-
 	bio = bio_alloc_bioset(NULL,
 			       bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS),
 			       REQ_OP_READ,
@@ -158,8 +154,6 @@ static int bch2_direct_IO_read(struct kiocb *req, struct iov_iter *iter)
 		bch2_read(c, rbio_init(bio, opts), inode_inum(inode));
 	}
 
-	iter->count += shorten;
-
 	if (sync) {
 		closure_sync(&dio->cl);
 		closure_debug_destroy(&dio->cl);
Signature · 1 line
2.42.0
#1 lore raw
Quoted 8 lines · Brian Foster

bch2_direct_IO_read() checks the request offset and size for sector alignment and then falls through to a couple calculations to shrink the size of the request based on the inode size. The problem is that these checks round up to the fs block size, which runs the risk of underflowing iter->count if the block size happens to be large enough. This is triggered by fstest generic/361 with a 4k block size, which subsequently leads to a crash.

Quoted 4 lines

After some discussion, the original purpose of the shorten logic in this path is not totally clear. It appears to be intended as an optimization of limited value, so simplify things and avoid the underflow problem by removing it.

Thinking about this a bit more...

All that we're avoiding with the shorten is zeroing out the rest of the read, which will happen in bch2_read_extent(); we won't be issuing any IO, since that would require real data extents above i_size, which is of course not allowed.

But zeroing out the rest of the read is actual work that's being avoided, since if we return a short read (which is what we're doing here) it's undefined what happens to the rest of the buffer.

So... maybe we should keep it, for the crazy but inevitable application that uses a 1MB read buffer for everything, including O_DIRECT reads to 4k files?

#2 lore raw
Quoted 21 lines · Kent Overstreet
Quoted 8 lines · Brian Foster

bch2_direct_IO_read() checks the request offset and size for sector alignment and then falls through to a couple calculations to shrink the size of the request based on the inode size. The problem is that these checks round up to the fs block size, which runs the risk of underflowing iter->count if the block size happens to be large enough. This is triggered by fstest generic/361 with a 4k block size, which subsequently leads to a crash.

Quoted 4 lines

After some discussion, the original purpose of the shorten logic in this path is not totally clear. It appears to be intended as an optimization of limited value, so simplify things and avoid the underflow problem by removing it.

Thinking about this a bit more...

All that we're avoiding with the shorten is zeroing out the rest of the read, which will happen in bch2_read_extent(); we won't be issuing any IO, since that would require real data extents above i_size, which is of course not allowed.

Quoted 8 lines

But zeroing out the rest of the read is actual work that's being avoided, since if we return a short read (which is what we're doing here) it's undefined what happens to the rest of the buffer.

So... maybe we should keep it, for the crazy but inevitable application that uses a 1MB read buffer for everything, including O_DIRECT reads to 4k files?

I had another change around that basically just protected against the underflow. I.e., something along the lines of the following, to also cover the additional use of shorten later in the function:

	if (shorten >= iter->count)
		shorten = 0;

... and IIRC that seemed to work as well. But I'll probably have to take a closer look at that large buffer case to grok what's happening and the original purpose of this logic.

Brian

#3 lore raw
Quoted 31 lines · Brian Foster
Quoted 21 lines · Kent Overstreet
Quoted 8 lines · Brian Foster

bch2_direct_IO_read() checks the request offset and size for sector alignment and then falls through to a couple calculations to shrink the size of the request based on the inode size. The problem is that these checks round up to the fs block size, which runs the risk of underflowing iter->count if the block size happens to be large enough. This is triggered by fstest generic/361 with a 4k block size, which subsequently leads to a crash.

Quoted 4 lines

After some discussion, the original purpose of the shorten logic in this path is not totally clear. It appears to be intended as an optimization of limited value, so simplify things and avoid the underflow problem by removing it.

Thinking about this a bit more...

All that we're avoiding with the shorten is zeroing out the rest of the read, which will happen in bch2_read_extent(); we won't be issuing any IO, since that would require real data extents above i_size, which is of course not allowed.

Quoted 8 lines

But zeroing out the rest of the read is actual work that's being avoided, since if we return a short read (which is what we're doing here) it's undefined what happens to the rest of the buffer.

So... maybe we should keep it, for the crazy but inevitable application that uses a 1MB read buffer for everything, including O_DIRECT reads to 4k files?

Quoted 7 lines

I had another change around that basically just protected against the underflow. I.e., something along the lines of the following, to also cover the additional use of shorten later in the function:

	if (shorten >= iter->count)
		shorten = 0;

The change is a clean fix without breaking logic in bch2_direct_IO_read. If without shroten, the next loop counting on iter->count must be altered. I'd like the simpler way.

--
Su

Quoted 5 lines

... and IIRC that seemed to work as well. But I'll probably have to take a closer look at that large buffer case to grok what's happening and the original purpose of this logic.

Brian

#4 lore raw
Quoted 50 lines · Su Yue
Quoted 47 lines · Brian Foster
Quoted 35 lines · Kent Overstreet
Quoted 1 line · Brian Foster

bch2_direct_IO_read() checks the request offset and size for >

sector

Quoted 1 line

alignment and then falls through to a couple calculations to >

shrink

Quoted 1 line

the size of the request based on the inode size. The problem > is

that

Quoted 1 line

these checks round up to the fs block size, which runs the > risk

of

Quoted 5 lines

underflowing iter->count if the block size happens to be > large enough. This is triggered by fstest generic/361 with a 4k > block size, which subsequently leads to a crash.

After some discussion, the original purpose of the shorten > logic

in this

Quoted 1 line

path is not totally clear. It appears to be intended as an >

optimization

Quoted 1 line

of limited value, so simplify things and avoid the underflow >

problem by

Quoted 1 line

removing it.

Thinking about this a bit more...

All that we're avoiding with the shorten is zeroing out the rest of the
read, which will happen in bch2_read_extent(); we won't be issuing any
IO, since that would require real data extents above i_size, which is of
course not allowed.

But zeroing out the rest of the read is actual work that's being avoided, since if we return a short read (which is what we're doing here) it's undefined what happens to the rest of the buffer.

Quoted 6 lines

So... maybe we should keep it, for the crazy but inevitable application
that uses a 1MB read buffer for everything, including O_DIRECT reads to
4k files?

I had another change around that basically just protected against the underflow. I.e., something along the lines of the following, to also cover the additional use of shorten later in the function:

Quoted 8 lines
Quoted 3 lines
	if (shorten >= iter->count)
		shorten = 0;

The change is a clean fix without breaking logic in bch2_direct_IO_read. If without shroten, the next loop counting on iter->count must be altered. I'd like the simpler way.

Hi Su,

I'm not quite following your comment on the "next loop counting on iter->count." Can you elaborate? Also, which option are you calling more simple? :) Thanks.

Brian

Quoted 9 lines

--
Su

Quoted 5 lines

... and IIRC that seemed to work as well. But I'll probably have to take a closer look at that large buffer case to grok what's happening and the original purpose of this logic.

Brian

#5 lore raw
Quoted 49 lines · Brian Foster
Quoted 48 lines · Su Yue
Quoted 45 lines · Brian Foster
Quoted 43 lines · Kent Overstreet
Quoted 1 line · Brian Foster

bch2_direct_IO_read() checks the request offset and size for >

sector

Quoted 1 line

alignment and then falls through to a couple calculations to >

shrink

Quoted 1 line

the size of the request based on the inode size. The problem > is

that

Quoted 1 line

these checks round up to the fs block size, which runs the > risk

of

Quoted 7 lines

underflowing iter->count if the block size happens to be

Quoted 1 line

large

enough. This is triggered by fstest generic/361 with a 4k

Quoted 1 line

block

size, which subsequently leads to a crash.

After some discussion, the original purpose of the shorten > logic

in this

Quoted 1 line

path is not totally clear. It appears to be intended as an >

optimization

Quoted 1 line

of limited value, so simplify things and avoid the underflow >

problem by

Quoted 1 line

removing it.

Thinking about this a bit more...

All that we're avoiding with the shorten is zeroing out the rest of the
read, which will happen in bch2_read_extent(); we won't be issuing any
IO, since that would require real data extents above i_size, which is of
course not allowed.

But zeroing out the rest of the read is actual work that's being avoided, since if we return a short read (which is what we're doing here) it's undefined what happens to the rest of the buffer.

So... maybe we should keep it, for the crazy but inevitable application
that uses a 1MB read buffer for everything, including O_DIRECT reads to
4k files?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

I had another change around that basically just protected against the underflow. I.e., something along the lines of the following, to also cover the additional use of shorten later in the function:

	if (shorten >= iter->count)
		shorten = 0;

Sorry for the misleading reply.
I mean the check of shorten is efficient and simpler which looks good to me.

Quoted 6 lines
Quoted 5 lines

The change is a clean fix without breaking logic in bch2_direct_IO_read. If without shroten, the next loop counting on iter->count must be altered. I'd like the simpler way.

Quoted 6 lines

Hi Su,

I'm not quite following your comment on the "next loop counting on iter->count." Can you elaborate? Also, which option are you calling more simple? :) Thanks.

I think I just explained Kent's opinion in another way. The loop is

fs/bcachefs/fs-io-direct.c:
static int bch2_direct_IO_read(struct kiocb *req, struct iov_iter *iter) {
91 iter->count -= shorten;
...

130         while (iter->count) {
               ...
               bch2_read()
           }

           iter->count += shorten;
}

bch2_direct_IO_read() does all boundary check for dio. If shorten is removed and no other condition in the loop, it brings more bios, more btree searches in bch2_read[_extent].

--
Su

Quoted 11 lines

Brian

Quoted 9 lines

--
Su

Quoted 5 lines

... and IIRC that seemed to work as well. But I'll probably have to take a closer look at that large buffer case to grok what's happening and the original purpose of this logic.

Brian